Stereoscopic image display device, terminal device, stereoscopic image display method, and program thereof

ABSTRACT

The stereoscopic image display device which displays images corresponding to each of a plurality of viewpoints includes: a stereoscopic image display panel which includes a display panel in which a plurality of pixels are arranged and a light-ray separating module provided on the display panel for separating parallax images from each of the pixels towards a plurality of N-viewpoints (N is a natural number of 2 or larger) according to the layout direction of each of the pixels; an observer position measuring unit which measures an observing position of the observer who is facing the display surface; a relative position calculating unit which calculates a relative position of the observer with respect to the stereoscopic image display panel based on the measurement result; and an image generation processing unit which generates viewpoint image by corresponding to the relative position and outputs the image towards the stereoscopic image display panel.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromJapanese patent applications No. 2013-200197, filed on Sep. 26, 2013,and No. 2014-127895, filed on Jun. 23, 2014, the disclosure of which isincorporated herein in its entirety by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a stereoscopic image display device.More specifically, the present invention relates to a stereoscopic imagedisplay device which displays images that correspond to each of aplurality of viewpoints, a terminal device that is provided with thestereoscopic image display device, a stereoscopic image display method,and a program thereof.

2. Description of the Related Art

Conventionally, there is a stereoscopic image display devices capable ofachieving stereopsis without using eyeglasses through providing imagesdifferent from each other towards a plurality of viewpoints andprojecting images of different parallax to the left and right eyes ofthe observer.

As a stereoscopic image display method employed with such device, knownis a method which: synthesizes image data for each of the viewpoints anddisplays it on a display unit; separates the displayed synthesized imageby an optical separating module constituted with a lens or a barrier (alight-shielding plate) having a slit; and provides those to each of theviewpoints of the observer.

As the optical separating device, generally employed are a parallaxbarrier having a great number of striped slits and a lenticular lens inwhich cylindrical lenses having the lens effect in one direction arearranged.

With the stereoscopic image display device employing such displaymethod, it is necessary to project appropriate parallax images to eachof the left and right eyes of the observer. Thus, the space region(stereopsis region) where the observer can properly observe thestereoscopic image is limited. Therefore, when the positions of the leftand right eyes of the observer are shifted out of the stereopsis region,the appropriate stereoscopic image cannot be visually recognized due tothe existence of a crosstalk region where the parallax images overlapwith each other so that a double image is visually recognized and areverse view region where each of the parallax images is projectedinversely.

As a method for improving such issue, there is a method which detectspositions of the eyes of the observer and executes switching control ofparallax images by using the detection result, and known is a displaydevice in which the visibility of the stereoscopic images is improved inthe crosstalk region and the reverse view region by employing thatmethod (e.g., Japanese Patent Publication H 10-333091 (Patent Document1), Japanese Unexamined Patent Publication 2011-215499 (Patent Document2), Japanese Unexamined Patent Publication 2012-120194 (Patent Document3), Japanese Unexamined Patent Publication 2012-60607 (Patent Document4), and Japanese Unexamined Patent Publication 2005-91447 (PatentDocument 5)).

Patent Document 1 discloses a technical content which executes parallaxbarrier control and image switching control in a stereoscopic imagedisplay device in which the resolution of the parallax barrier isdoubled, when the observer is in a crosstalk region or a reverse viewregion. Further, the display device disclosed in Patent Document 2includes: a light source unit having definition of four times of 3Dresolution on the backlight side; and a light source control unit forcontrolling the light source unit. Disclosed therein is a technicalcontent with which the light source control unit executes switchingcontrol of the functions of the light source that constitutes the lightsource unit according to the position of the observer.

That is, disclosed in Patent Documents 1 and 2 are the display deviceswhich present images of less sense of uncomfortable feeling according tothe detected position of the observer through having a light-raydistribution module exceeding the resolution of the display panel. Inthose display devices, the number of device viewpoints (the number ofregions where the so-called parallax images are to be projected)determined according to the relation between the display panel and thelight-ray distribution module is increased to take a countermeasure forreverse view.

In the image display device disclosed in Patent Document 3, a sense ofuncomfortableness felt at the time of viewing is suppressed with atechnique which sets off the parallax barrier function and outputs 2Dimages when it is detected that there are a plurality of observers.Further, in the stereoscopic image display device disclosed in PatentDocument 4, a sense of uncomfortableness is lightened with a techniquewhich controls the parallax amount according to the position of theobserver.

Patent Document 5 discloses a technical content with which a lobecontrol module constituted by including a parallax barrier executesswitching control of the stereopsis region to achieve multi-viewpointsby using side lobes even in a case where tracking is not used. Further,in the stereoscopic display device, a sense of uncomfortableness feltwhen one of the eyes of the observer is located in the main lobe and theother eye is located in the side lobe through a technique that is acombination of tracking and the processing by the lobe control module.

However, there are following issues with the display devices accordingto each of Patent Documents described above which employ the switchingcontrol of the parallax images based on the detected result regardingthe positions of both eyes of the observer.

With the display device disclosed in Patent Document 1 or 2, ahigh-definition light-ray separating module having the resolution oftwice or more of the display panel is required. This causes increase inthe cost for the structure or manufacture. Further, while prescribedimage processing can be performed within the range of the number of theviewpoints of the device, it is only possible to present repeated videosto be visually recognized for the viewing angles over the number ofviewpoints of the device.

Further, Patent Documents 3 and 4 disclose the image display deviceemploying the structure that makes it possible to effectively workwithin the number of viewpoints of the device. That is, no technicalmeasure for suppressing the deterioration of the image quality in thereverse view region is employed, so that a sense of uncomfortableness isto be felt when visually recognizing the images in the reverse viewregion. Furthermore, a parallax barrier is provided as a lobe controlmodule in Patent Document 5, so that the transmittance is low.

It is an exemplary object of the present invention to improve theshortcomings of the related techniques and, more specifically, toprovide a stereoscopic image display device and a terminal device, whichgenerate and display significant parallax images corresponding to theposition of the observer.

DISCLOSURE OF THE INVENTION

In order to achieve the foregoing object, the stereoscopic image displaydevice employs a structure which includes: a stereoscopic image displaypanel including a display panel in which a plurality of pixels arearranged and a light-ray separating module provided on a display surfaceside of the display panel for separating parallax images from each ofthe pixels towards a plurality of N-viewpoints (N is a natural number of2 or larger) according to layout direction of each of the pixels; anobserver position measuring unit which measures an observing position ofan observer who is facing the display surface; a relative positioncalculating unit which calculates a relative position of the observerwith respect to the stereoscopic image display panel based on a resultof the measurement; and an image generation processing unit which has afunction for generating viewpoint images for J-viewpoints (J>N) set inadvance, and generates one viewpoint image or more according to each ofthe viewpoints constituting the N-viewpoints by corresponding to therelative position and outputs the viewpoint image towards thestereoscopic image display panel.

Further, the terminal device according to the present invention employsa structure which includes the stereoscopic image display device whichdisplays images corresponding to each of a plurality of viewpoints, anda casing which houses the stereoscopic image display device insidethereof.

The stereoscopic image display method according to the present inventionis used with a stereoscopic image display device which includes: astereoscopic image display panel including a display panel in which aplurality of pixels are arranged and a light-ray separating moduleprovided on a display surface side of the display panel for separatingparallax images from each of the pixels towards a plurality ofN-viewpoints (N is a natural number of 2 or larger) according to layoutdirection of each of the pixels; an observer position measuring unitwhich measures an observing position of an observer who is facing thedisplay surface; and a display controller which generates the parallaximages according to a result of the measurement, and the method ischaracterized that: the observer position measuring unit measures anobserving position of the observer; the display controller calculates arelative position of the observer with respect to the stereoscopic imagedisplay panel based on the result of the measurement; the displaycontroller generates viewpoint images for J-viewpoints (J>N) set inadvance; the display controller specifies a plurality of viewpointimages according to the relative position from the generated viewpointimages; and the display controller outputs the specified plurality ofviewpoint images towards the stereoscopic image display panel.

Further, the stereoscopic image display program according to the presentinvention is used with a stereoscopic image display device whichincludes: a stereoscopic image display panel including a display panelin which a plurality of pixels are arranged and a light-ray separatingmodule provided on a display surface side of the display panel forseparating parallax images from each of the pixels towards a pluralityof N-viewpoints (N is a natural number of 2 or larger) according tolayout direction of each of the pixels; an observer position measuringunit which measures an observing position of an observer who is facingthe display surface; and a display controller which generates theparallax images according to a result of the measurement, and theprogram causes a computer provided in advance to the display controllerto function as: an observer position measuring module which measures anobserving position of the observer; an observer position calculatingmodule which calculates a relative position of the observer with respectto the stereoscopic image display panel based on the result of themeasurement; and an image generation processing module which generatesviewpoint images for J-viewpoints (J>N) set in advance, specifies aplurality of viewpoint images according to the relative position fromthe generated viewpoint images, and outputs the specified plurality ofviewpoint images towards the stereoscopic image display panel.

BRIEF DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a block diagram showing functional structures of astereoscopic image display device according to a first exemplaryembodiment of the present invention;

FIG. 2 is a schematic view showing a structural example of thestereoscopic image display device disclosed in FIG. 1;

FIGS. 3A and 3B show examples of image information of two viewpointsinputted to pixels of each of the viewpoints of the stereoscopic imagedisplay device disclosed in FIG. 1, in which FIG. 3A is afirst-viewpoint image and FIG. 3B is a second-viewpoint image;

FIG. 4 is a chart showing an example of a method for generating thefirst-viewpoint image and the second-viewpoint image shown in FIGS. 3Aand 3B;

FIG. 5 is a chart showing a state of light-ray separation and astereopsis region by a stereoscopic image display panel disclosed inFIG. 1;

FIG. 6 is a chart showing a state of light-ray separation and a reverseview region by the stereoscopic image display panel disclosed in FIG. 1;

FIGS. 7A-7F show examples of image information inputted to thestereoscopic image display device of two viewpoints disclosed in FIG. 1,in which FIG. 7A is an image corresponding to a first-viewpoint image,FIG. 7B is an image corresponding to a second-viewpoint image, FIG. 7Cis an image corresponding to a third-viewpoint image, FIG. 7D is animage corresponding to a fourth-viewpoint image, FIG. 7E is an imagecorresponding to a fifth-viewpoint image, and FIG. 7F is an imagecorresponding to a sixth-viewpoint image,

FIG. 8 is a chart showing an example of a method for generating thefirst-viewpoint image to the sixth-viewpoint image shown in FIGS. 7A-7F;

FIGS. 9A-9C show the states regarding the relevancy between the state oflight-ray separation done by the stereoscopic image display panel andthe positions of the both eyes of the observer, in which FIG. 9A is acase where the left eye is in a first-viewpoint region 50 a and theright eye is in a second-viewpoint region 50 b, FIG. 9B is a case wherethe left eye is in a first-viewpoint region 51 b and the right eye is ina second-viewpoint region 50 a, and FIG. 9C is a case where the left eyeis in a first-viewpoint region 51 a and the right eye is in asecond-viewpoint region 51 b;

FIG. 10 is an enlarged chart of FIG. 9C, showing an example of each ofsub-regions acquired by dividing the first-viewpoint region into fourregions;

FIG. 11 is a table showing the corresponding relation between theviewpoint regions where the left and right eyes of the observer arelocated and the image data to be displayed in the first-viewpoint regionand the second-viewpoint region based on each of the stereopsis regiondisclosed in FIG. 9;

FIG. 12 is a chart showing a state of light-ray separation done by atypical 6-viewpoint stereoscopic image display panel;

FIG. 13 is a chart showing a state of light-ray separation done by a3-viewpoint stereoscopic image display panel according to the firstexemplary embodiment;

FIG. 14 is a chart showing an example of image data inputted to the3-viewpoint stereoscopic image display panel shown in FIG. 13 and thestereoscopic image display panels of the second and third exemplaryembodiments;

FIG. 15 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst to third viewpoint regions when the observer moves to aminus-direction side of the x-axis as shown in FIG. 13;

FIG. 16 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst to third viewpoint regions when the observer moves to aplus-direction side of the x-axis as shown in FIG. 13;

FIG. 17 is a chart showing a state of light-ray separation done by a4-viewpoint stereoscopic image display panel according to the firstexemplary embodiment;

FIG. 18 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst to fourth viewpoint regions when the observer moves to aminus-direction side of the x-axis as shown in FIG. 17;

FIG. 19 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst to fourth viewpoint regions when the observer moves to aplus-direction side of the x-axis as shown in FIG. 17;

FIG. 20 is a flowchart showing operations of the stereoscopic imagedisplay device disclosed in FIG. 1;

FIG. 21 is a chart showing a state of light-ray separation done by a2-viewpoint stereoscopic image display panel according to the secondexemplary embodiment;

FIG. 22 is a chart showing the 3D crosstalk characteristic in the centerpart of the display surface of the 2-viewpoint stereoscopic imagedisplay device disclosed in FIG. 21;

FIG. 23 is a chart showing the viewpoint regions where the left andright eyes are located when the observer moves to the minus side in thex-axis direction from the state disclosed in FIG. 22;

FIG. 24 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst-viewpoint region as well as the second-viewpoint region based onthe positions of the observer disclosed in FIG. 23;

FIG. 25 is a chart showing the viewpoint regions where the left andright eyes of the observer are located in a case of using a stereoscopicimage display panel having a relation of “d<e (d: distance between botheyes of the observer, e: viewpoint pitch regarding the first-viewpointregion and the second-viewpoint region)” as a comparison example of FIG.23;

FIG. 26 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst-viewpoint region as well as the second-viewpoint region based onthe positions of the observer disclosed in FIG. 25;

FIG. 27 is a chart showing a state of light-ray separation done by a4-viewpoint stereoscopic image display panel according to the thirdexemplary embodiment;

FIG. 28 is a chart showing the relation between the observer and theviewpoint regions when the observer moves to a minus-direction side ofthe x-axis;

FIG. 29 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst to fourth viewpoint regions based on the positions of the observershown in FIG. 28;

FIG. 30 is a schematic view showing the structure of a stereoscopicimage display panel according to a fourth embodiment of the presentinvention;

FIGS. 31A and 31B show charts of pixels in which sub-pixels constitutedwith four primary colors are arranged in matrix, in which FIG. 31A is achart showing a layout relation between a pixel constituted with 4×4sub-pixels and a lens element and FIG. 31B shows a state where thepixels and the lens elements disclosed in FIG. 31A are arranged inmatrix;

FIG. 32 is a chart showing the pixel constituted with 4×4 sub-pixels andthe lens array element in a first layout state shown in FIG. 31A;

FIG. 33 is a chart showing the relation between an input image andsub-pixels (a sub-pixel group) in a case where the pixel constitutedwith 4×4 sub-pixels and the lens array element disclosed in FIG. 32 arerotated clockwise by 90 degrees;

FIG. 34 is a chart showing light-ray separation when an image isseparated only in one direction by using an active light-ray separatingmodule corresponding to the pixel constituted with 4×4 sub-pixelsdisclosed in FIG. 31A;

FIG. 35 is a chart showing the relation between the position of theobserver and the 3D crosstalk characteristic corresponding to FIG. 34;

FIG. 36 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst-viewpoint region as well as the second-viewpoint region based onthe positions of the observer shown in FIG. 35;

FIGS. 37A and 37B show charts regarding the relations based on the pixelconstituted with 4'4 sub-pixels and the lens array element disclosed inFIG. 31A, in which FIG. 37A is a chart showing the relation between aninput image and sub-pixels (a sub-pixel group) in a first layout stateand FIG. 37B is a chart showing the relation between an input image andsub-pixels (a sub-pixel group) in a second layout state;

FIG. 38 is a chart showing an example of image information inputted to astereoscopic image display panel of 2×2 viewpoints having pixelsdisclosed in FIG. 37;

FIG. 39 is a chart showing light-ray separation when an image isseparated in two directions by using an active light-ray separatingmodule corresponding to the pixel constituted with 4×4 sub-pixelsdisclosed in FIG. 31A;

FIG. 40 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst-viewpoint region as well as the second-viewpoint region based onthe positions of the observer disclosed in FIG. 39;

FIG. 41 is a schematic view showing a terminal device which includes astereoscopic image display device according to a fourth exemplaryembodiment of the present invention;

FIG. 42 is a block diagram showing the functional structures of thestereoscopic image display device according to the fourth exemplaryembodiment of the present invention;

FIG. 43 is a flowchart showing operations of the stereoscopic imagedisplay device disclosed in FIG. 42;

FIG. 44 is a chart showing a state where images corresponding to each ofthe two directions according to the position of the observer arepresented to the observer by using the stereoscopic image display devicehaving the unidirectional light-ray separating module disclosed in FIG.2; and

FIG. 45 is a table showing the relation between the viewpoint regionswhere each of the left and right eyes is located and image data of thefirst-viewpoint region as well as the second-viewpoint region bycorresponding to each of the positions of the observer disclosed in FIG.44.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First ExemplaryEmbodiment

A first exemplary embodiment of the stereoscopic image display deviceaccording to the present invention will be described by referring toFIG. 1 to FIG. 20.

(Overall Structures)

As shown in FIG. 2, a stereoscopic image display device 1 which displaysimages corresponding to each of a plurality of viewpoints includes: anobserver position measuring unit 45 which is provided with a cameracapable of capturing visible light and infrared rays for measuring theposition of an observer 100; a display controller (not shown) whichgenerates 3D data based on a measurement result acquired from theobserver position measuring unit 45; and a stereoscopic image displaypanel 10 which separates an image and emits those based on the 3D dataand a control signal outputted from the display controller.

The stereoscopic image display panel 10 includes: a display panel 11 inwhich pixels each containing at least a sub-pixel for displaying afirst-viewpoint image and a sub-pixel for displaying a second-viewpointimage are arranged in matrix; and a light-ray separating module 12 whichis provided on the display surface side of the display panel 11 forseparating parallax images from the pixels towards each of theviewpoints.

The light-ray separating module 12 employs a structure in which a lens15 corresponding to each of the pixels arranged on the display panel 11is arranged in an array form, so that images for each of the viewpointscan be separated to prescribed different positions.

With such structure, the stereoscopic image display panel 10 can projectdifferent images towards at least two viewpoints, i.e., can executelight-ray separation. Regarding the direction of the light-rayseparation, as shown in FIG. 2, it is defined that the layout directionof the lenses 15 (a first pixel layout direction) is the x-axis, adirection which is orthogonal to the x-axis and extended to the observer100 from the display panel 11 is the z-axis, and a direction orthogonalto both the x-axis and the z-axis (a second pixel layout direction) isthe y-axis. Further, the intersection point (origin) of each of thoseaxes is defined to be located in the center of the stereoscopic imagedisplay panel 10. Hereinafter, it is also called as the origin (thecenter of the display surface) of the stereoscopic image display panel10. In that case, the straight line connecting the left eye 101 and theright eye 102 of the observer 100 is substantially in parallel to thex-axis.

The observer position measuring unit 45 is structured to capture theobserver 100 by the camera provided to itself, and to measure thepositions of the left eye 101 and the right eye 102 of the observer 100based thereupon.

When the observer position measuring unit 45 measures the positions ofthe both eyes (the left eye 101 and the right eye 102), various methodscan be used.

For example, it is possible to employ: a method which measures theposition in the x-axis direction and the y-axis direction throughdetecting the positions of the face and the eyes by using patternmatching from an image captured by visible light, and measures thez-axis position based on the distance between the eyes derived from themeasurement result; a method called a TOF method which irradiatesinfrared sine wave light to the observer 100 and measures the positionin the z-axis direction from the time difference (phase difference) ofthe light flight until the sine wave light that is the reflection lightof the irradiated light arrives at the camera; etc.

As the display panel 11, it is possible to use various electro-opticalelements such as a liquid crystal display element, an organicelectroluminescence display element, an electrophoretic element, or anelectrochromic element.

As the light-ray separating module 12, it is possible to use not only anoptical element such as a lenticular lens, a fly-eye lens, or a parallaxbarrier but also an electro-optical element such as a liquid crystallens having a refractive index control function or a liquid crystalbarrier having a light-shield control function. In view of thetransmittance, it is preferable to use a lenticular lens, a fly-eyelens, and a liquid crystal lens.

First, for simplifying the explanation, a case of employing a2-viewpoint stereoscopic image display panel as the stereoscopic imagedisplay panel 10 will be described. Note here that an N-viewpointstereoscopic image display panel (N is a natural number of 2 or larger)is a stereoscopic image display panel which separates and emitstwo-dimensional image data having parallax for stereoscopic displaytowards N-viewpoints.

FIGS. 3A and 3B show examples of 2-viewpoint image information inputtedto the pixels of each viewpoint of the stereoscopic image display panel10. FIG. 3A is a first-viewpoint image 60 a, and FIG. 3B is asecond-viewpoint image 60 b.

Between each of the viewpoint images, the size of the faces of “5” ofdices and the size of “3” are different, respectively. That is, as shownin FIG. 3, employed is a structure in which the face of “5” isrelatively large in FIG. 3A which corresponds to the first-viewpointimage 60 a and the face of “3” is relatively large in FIG. 3B whichcorresponds to the first-viewpoint image 60 b.

Then, FIG. 4 shows an example of a method for generating thefirst-viewpoint image and the second-viewpoint image shown in FIGS. 3Aand 3B. FIG. 4 is an explanatory chart regarding an image capturingmethod called an intersection method. Herein, the intersection point 66between each of the optical axes of a firs-viewpoint camera 65 a and asecond-viewpoint camera 65 b is set on a normal screen, i.e., on areproduction position on a screen surface. Further, it is defined thatthe horizontal direction of the screen surface is the x-axis, thedirection orthogonal to the x-axis is the y-axis (not shown), thedirection orthogonal to the xy plane is the z-axis, and the intersectionpoint 66 is the origin at which each of the axes intersects with eachother, and the plus and minus directions in the x-axis and the z-axisare as shown in FIG. 4. Note here that the layout positions of thefirst-viewpoint camera 65 a and the second-viewpoint camera 65 b are setto take a same value on the y-axis (set to have a same coordinate valueon the y-axis: to be in a horizontal state).

Through capturing a 3D object 64 of a dice having three-dimensionalinformation of xyz values in that state by the first-viewpoint andsecond-viewpoint cameras (65 a, 65 b), the images as shown in FIG. 3Aand FIG. 3B are generated.

Between each of the images captured and generated in this manner,positions of the mutually corresponding points (corresponding points)are different as shown in FIGS. 3A and 3B. That is, the correspondingpoints in FIG. 3A and FIG. 3B (e.g., each of the vertex points of thedices) are different in both images, and the differences in the size arethe parallax amount. The extent of the parallax amount is determineddepending on the camera positions (x, z values), the angles of view ofthe cameras, distance between the cameras, the 3D object position (zvalue), and the like.

Subsequently, FIG. 5 shows the state of light-ray separation done by thestereoscopic image display panel 10. This is a sectional view of a zxplane when viewed from the plus direction of the y-axis. In FIG. 5, theleft eye of the observer 100 is expressed as “L” and the right eye isexpressed as “R”. This is the same for each of the following charts.

By a combination of the pixels for each of the above-describedviewpoints and the light-ray separating module 12, an image is separatedin such a manner that the first-viewpoint regions 50 a, 51 a, 52a, - - - and the second-viewpoint regions 50 b, 51 b, 52 b, - - - arealternately repeated along the x-axis direction.

When it is defined that each of the first-viewpoint regions (50 a, 51 a,52 a) is the left-eye region and each of the second-viewpoint regions(50 b, 51 b, 52 b) is the right-eye region and it is assumed that thefirst-viewpoint image 60 a is projected to each of the first-viewpointregions (FIGS. 3A and 3B) and the second-viewpoint image 60 b isprojected to each of the second-viewpoint regions (FIGS. 3A and 3B), thestereoscopic image display panel 10 is structured to form a stereopsisregion 80 where the observer 100 can sense the stereoscopic images whenthe left eye 101 is located in the left-eye region 50 a and the righteye 102 is located in the right-eye region 50 b, respectively.

The observer 100 herein is in a state isolated from the stereoscopicimage display panel 10 by the length of an optimum observing distance(optimum viewing distance) OD at which the stereoscopic region becomesthe maximum.

Reference code “e” shows a viewpoint pitch between each of the viewpointregions at the optimum viewing distance, and FIG. 5 shows a case wherethe viewpoint pitch e is equivalent to the distance d between the botheyes. Further, reference numerals “1” and “2” correspond to thefirst-viewpoint image 60 a and the second-viewpoint image 60 b,respectively.

Similarly, the stereoscopic image display panel 10 forms a stereopsisregion 81 when the both eyes of the observer 100 correspond to theleft-eye region 51 a and the right-eye region 51 b and forms astereopsis region 82 when the both eyes of the observer 100 correspondto the left-eye region 52 a and the right-eye region 52 b based onsignals or the like from a display controller.

Note here that the stereopsis region 80 including the z-axis extendedvertically from the center of the display surface (the center of theimage display) of the stereoscopic image display panel 10 is defined asthe main lobe, and the stereoscopic regions 81, 82 which appear when theobserver 100 moves in each of the directions on the plus side and theminus side on the x-axis from the main lobe are defined as the sidelobes.

Further, other than the above-described two stereopsis regions, it isalso possible to structure the stereoscopic image display panel 10 insuch a manner that a single or a plurality of side lobes correspondingto the position of the observer 100 appear on the minus-direction sideon the x-axis of the stereopsis region 81 and on the plus-direction sideon the x-axis of the stereoscopic region 82, respectively.

In a region on the way along which the observer 100 moves to the sidelobe from the main lobe, as shown in FIG. 6, the observer 100 comes tobe in a reverse view state where the right-eye region corresponds to theleft eye 101 of the observer 100 and the left-eye region corresponds tothe right eye 102, respectively. Thus, a reverse view region 85 isformed therein by the stereoscopic image display panel 10. The observer100 in that case is in a state where the left eye 101 senses thesecond-viewpoint image 60 b (2) and the right eye 102 senses thefirst-viewpoint image 60 a (1).

Then, each functional structure of the stereoscopic image display device1 according to the first exemplary embodiment is shown in FIG. 1. Asshown in the block diagram, along with the display panel 11, thelight-ray separating module 12, and the observer position measuring unit45 described above, the stereoscopic image display device 1 includes adisplay controller 24 which has a function for driving the display panel11 and a function for controlling the light-ray separating module 12.

The display controller 24 includes: a relative position calculating unit40 which calculates the relative position of the left eye 101 and theright eye 102 of the observer 100 with respect to the stereoscopic imagedisplay panel 10 based on the measurement result (measurementinformation) acquired by the observer position measuring unit 45; animage generation processing unit 30 which generates 3D data (parallaximages) and control signals based on the positional information(relative positional information) from the relative position calculatingunit 40; a display panel driving circuit (display panel driving unit) 21which drives the display panel 11 based on the 3D data from the imagegeneration processing unit 30; and an image distribution control circuit(image distribution control unit) 22 which controls operations of thelight-ray separating module 12 based on the control signals from theimage generation processing unit 30.

More specifically, the relative position calculating unit 40 has afunction which calculates coordinate values (x1, y1, z1) showing therelative position of the left eye 101 of the observer with respect tothe origin of the stereoscopic image display panel 10 and coordinatevalues (x2, y2, z2) showing the relative position of the right eye 102of the observer with respect to the origin based on the measurementinformation from the observer position measuring unit 45 (see FIG. 2).Further, it is also possible to provide a functional structure forcalculating the viewing angle θ1 (=a tan (x1/z1)) of the left eye 101,for example, to the relative position calculating unit 40 based on thecalculated values.

The image generation processing unit 30 is constituted with: anarithmetic calculator 31 which executes image processing; a dataaccumulation unit 32 which stores display target data to be displayed onthe display panel 11; a memory 33 which stores an operation controlprogram and the like of the arithmetic calculator 31; and an external IF(interface) 34 which mediates exchange of information with outside.Because each of those structural members effectively functions, theimage generation processing unit 30 can generate significant image data(3D data) corresponding to each viewpoint according to the signalsreceived from the relative position calculating unit 40.

In addition to the operation control program of the arithmeticcalculator 31, information regarding the relative position of the lefteye 101 and the right eye 102 acquired from the relative positioncalculating unit 40, information regarding the stereopsis regions of thestereoscopic image display panel 10, and the like are stored in thememory 33. The arithmetic calculator 31 has a function (function forspecifying the viewpoint regions where the left and right eyes of theobserver are located) which judges which of the viewpoint regions theleft and right eyes of the observer 100 are located based on theinformation accumulated in the memory 33 and the information acquired asappropriate from a sensor (not shown) of the observer position measuringunit 45.

Generation of the image data in the image generation processing unit 30is achieved when the arithmetic calculator 31 that has read out thedisplay target data stored in the data accumulation unit 32 executesimage processing thereon. When the display target data isthree-dimensional data containing depth information, employed is apreferable method with which the calculator 31 executes renderingprocessing on the three-dimensional data to generate two-dimensionalimage data for each of the viewpoint having parallax (3D data used forstereoscopic display).

That is, the calculator 31 when generating 3D data used for stereoscopicdisplay is structured to have virtual cameras of prescribed viewpointnumbers for three-dimensional data and to execute rendering processingfor each of those. For example, the two dimensional image data for eachviewpoint having parallax as shown in FIGS. 3A and 3B is generatedthrough executing the rendering processing on each by the arithmeticcalculator 31 provided with the virtual cameras of prescribed number ofviewpoints (2-viewpoints herein) for the three-dimensional data.

Note here that it is preferable to use the method based on thethree-dimensional data containing depth information for generating theimage data by the image generation processing unit 30. However, it isalso possible to employ a structure with which the display target dataon which rendering processing has been executed in advance is stored inthe data accumulation unit 32 and it is selectively read out.

That is, it is possible to employ a method with which the display targetdata in a two-dimensional image data format corresponding to FIGS. 3Aand 3B is accumulated in advance to the data accumulation unit 32, andthe arithmetic calculator 31 selectively reads out the image datadepending on stereoscopic display or flat display. When such method isemployed, the rendering processing becomes unnecessary. Thus, it ispossible to employ the arithmetic calculator 31 of low processingcapacity and calculation speed than the case of using theabove-described generation method that requires the renderingprocessing. This makes it possible to acquire an advantage of being ableto constitute the image generation processing unit 30 at a low cost.

Further, the image generation processing unit 30 generates the 3D dataaccording to the positional information received from the relativeposition calculating unit 40, and outputs it to the display paneldriving circuit 21. Further, for outputting the 3D data, the imagegeneration processing unit 30 has a function which outputs controlsignals (signals for making the liquid crystal lens effective) generatedsimultaneously with that to the image distribution control circuit 22.

The display panel driving circuit 21 has a function for generatingsignals (synchronous signals and the like) required for driving thedisplay panel 11, and the image distribution control circuit 22 has afunction which generates signals for driving the light-ray separatingmodule 12.

Other than the liquid crystal lens, it is possible to use an activeelement such as a liquid crystal barrier element that can be controlledby electric signals as the light-ray separating module 12.

Further, it is also possible to use a static element such as alenticular lens, a fly-eye lens, a parallax barrier, a pin-hole, whichcannot be controlled by electric signals as the light-ray separatingmodule 12. With that, it becomes unnecessary to provide the imagedistribution control circuit 22 to the display controller 24, so that itis possible to decrease the cost and the like due to such simplificationof the structure.

Meanwhile, the image generation processing unit 30 employing theabove-described structure has a function for generating an image ofJ-viewpoints (J>N) set in advance based on the display-target datawithin the data accumulation unit 32.

The contents described heretofore can be summarized as follows. Thestereoscopic image display device 1 which displays images correspondingto each of a plurality of viewpoints includes: the stereoscopic imagedisplay panel 10 including the display panel 11 in which a plurality ofpixels are arranged and the light-ray separating module 12 provided onthe display surface side for separating parallax images from each of thepixels towards a plurality of N-viewpoints according to the layoutdirection (x-axis direction) of each of the pixels; the observerposition measuring unit 45 which measures the observing position of theobserver who is facing the display surface; the relative positioncalculating unit 40 which calculates the relative position of theobserver with respect to the stereoscopic image display panel 10 basedon the measurement result; and the image generation processing unit 30which has the function for generating viewpoint images for J-viewpoints(J>N) set in advance, and generates the viewpoint images corresponded tothe relative position and outputs those towards the stereoscopic imagedisplay panel.

Therefore, the image generation processing unit 30 can generate imagesfor more than 2-viewpoints such as images for 6-viewpoints, for example.

Now, as an example of the image information inputted to the 2-viewpointstereoscopic image display panel 10 according to the first exemplaryembodiment, images for 6-viewpoints are shown in FIGS. 7A-7F. FIG. 7A isa first-viewpoint image 61 a, FIG. 7B is a second-viewpoint image 61 b,FIG. 7C is a third-viewpoint image 61 c, FIG. 7D is a fourth-viewpointimage 61 d, FIG. 7E is a fifth-viewpoint image 61 e, and FIG. 7F is asixth-viewpoint image 61 f.

As shown in 7A-7F, the size of the faces of “5” and the faces of “3” ofthe dices is different for each of the viewpoints. That is, there issuch a characteristic that the face of “5” is the largest in thefirst-viewpoint image 61 a, while the face of “3” is the largest in thesixth-viewpoint image 61 f. Further, employed is a structure in whichthe size of the faces of “5” is gradually decreased while the size ofthe faces of “3” is gradually increased from FIG. 7A to FIG. 7F.

Then, FIG. 8 shows an example of a method for generating images from thefirst-viewpoint image to the sixth-viewpoint image shown in FIGS. 7A-7F.

Basically, as in the structural content shown in FIG. 4, thefirst-viewpoint camera 65 a, the second-viewpoint camera 65 b, thethird-viewpoint camera 65 c, the fourth-viewpoint camera 65 d, thefifth-viewpoint camera 65 e, and the sixth-viewpoint camera 65 f aredisposed, and a 3D object 64 of the dice having three-dimensionalinformation of xyz values is captured by the first to sixth-viewpointcameras (65 a to 65 f) to generate the images shown in FIG. 7A to FIG.7F. Further, the difference in the size in the positions among each ofthe corresponding points is the parallax amount among each of the imagesin FIGS. 7A-7F.

When the observer 100 moves to the outer side of the main lobe of thestereoscopic image display panel 10, the image generation processingunit 30 is structured to perform generation and the like of a new imagecorresponding to the viewpoints according to the observing direction.That is, in a case of generating the images for 6-viewpoints as shown inFIG. 7, the image generation processing unit 30 generates all the imagesfor 6-viewpoints and performs storing processing thereof in advance, andthe images may be selected as appropriate by corresponding to the moveof the observer. Alternatively, the image generation processing unit 30may be structured to generate only the images for the viewpoint in thevicinity of the position of the observer as appropriate.

The light-separation state done by the stereoscopic image display panel10 when the image generation processing unit 30 generates the images of6-viewpoints in the manner described above is shown in FIGS. 9A-9C. Asin the case of FIG. 5 described above, this is also a sectional view ofa zx plane when viewed from the plus direction of the y-axis (see FIG.2).

FIG. 9A is a chart showing an example of a scene where the imagegeneration processing unit 30 judges that the left eye 101 of theobserver 100 exists in the first-viewpoint region 50 a and the right eye102 exists in the second-viewpoint region 50 b from the calculationresult (positional information) acquired by the relative positioncalculating unit 40.

In this case, the image generation processing unit 30 generates theimage data of the third-viewpoint image 61 c (FIG. 7) for thefirst-viewpoint region 50 a and generates the image data of thefourth-viewpoint image 61 d (FIG. 7) for the second-viewpoint region 50b, respectively, and outputs those to the display panel driving circuit21 as the 3D data. That is, “3” and “4” applied therein correspond tothe third-viewpoint image 60 c and the fourth-viewpoint image 60 d,respectively.

Thereby, a stereopsis region 90 where the observer 100 can sense thestereoscopic images can be formed. As shown in FIGS. 9A-9C, thestereopsis region 90 as the main lobe in this case is formed at aposition that is roughly consistent with the stereopsis region 80 as themain lobe described by referring to FIG. 5.

Similarly, FIG. 9B is a chart showing an example of a scene where theimage generation processing unit 30 judges that the left eye 101 of theobserver 100 exists in the second-viewpoint region 51 b and the righteye 102 exists in the first-viewpoint region 50 a from the calculationresult acquired by the relative position calculating unit 40.

In this case, the image generation processing unit 30 generates theimage data of the second-viewpoint image 61 b (FIG. 7) for thesecond-viewpoint region 51 b and generates the image data of thethird-viewpoint image 61 c (FIG. 7) for the first-viewpoint region 50 a,respectively, and outputs those to the display panel driving circuit 21as the 3D data. That is, “2” and “3” applied therein correspond to thesecond-viewpoint image 60 b and the third-viewpoint image 60 c,respectively.

Thereby, a stereopsis region 91 where the observer 100 can sense thestereoscopic images can be formed. As in the case described above, theposition of the stereopsis region 91 is roughly consistent with theposition of the reverses view region 85 described by referring to FIG.6.

Similarly, FIG. 9C is a chart showing an example of a scene where theimage generation processing unit 30 judges that the left eye 101 of theobserver 100 exists in the first-viewpoint region 51 a and the right eye102 exists in the second-viewpoint region 51 b from the calculationresult acquired by the relative position calculating unit 40.

In this case, the image generation processing unit 30 generates theimage data of the first-viewpoint image 61 a (FIG. 7) for thefirst-viewpoint region 51 a and generates the image data of thesecond-viewpoint image 61 b (FIG. 7) for the second-viewpoint region 51b, respectively, and outputs those to the display panel driving circuit21 as the 3D data. That is, “1” and “2” applied therein correspond tothe first-viewpoint image 60 a and the second-viewpoint image 60 b,respectively.

Thereby, a stereopsis region 92 where the observer 100 can sense thestereoscopic images can be formed. As in the case described above, theposition of the stereopsis region 92 is roughly consistent with theposition of the side lobe 81 described by referring to FIG. 5.

Now, FIG. 11 shows a table showing the corresponding relation betweenthe viewpoint regions where the left and right eyes of the observer 100are located and the image data to be displayed in the first-viewpointregion and the second-viewpoint region defining that the stereopsisregions 92, 91, 90 and the two stereopsis regions generated on the plusdirection of the x-axis according to the same regularity thereof (FIGS.9A-9C) as the stereopsis regions A to E, respectively.

In the section of remarks in FIG. 11, the corresponding relationregarding the positions of each of the stereopsis regions A to E and thepositions of the stereopsis regions (main lobe and side lobes) as wellas the reverse view region shown in FIG. 5 and FIG. 6.

For displaying the images on the 2-viewpoint stereoscopic image displaypanel 10 based on the image data of 2-viewpoints, the observer 100 canonly sense the same video in the main lobe and the side lobes and areverse view is generated in the regions between the main lobe and theside lobes.

In the meantime, with the use of the above-described structure havingthe function of generating the images of 6-viewpoints for formingstereopsis regions corresponding to the observer 100 as appropriatebased on each of those images, the observer 100 can sense differentvideos in each of the stereopsis regions A to E even in the case of2-viewpoint stereoscopic display. That is, motion parallax is given,thereby making it possible to contribute to improving the sense ofambience in the stereoscopic image quality.

In general, as the stereoscopic image display device employing thestructure for giving motion parallax, known is a device which forms thestereopsis regions as shown in FIG. 12 by using a 6-viewpointstereoscopic image display panel. The stereopsis regions 92 a to 92 e inFIG. 12 are formed at positions corresponding to the stereopsis regionsA to E shown in FIG. 11. Further, “1”, “2”, “3”, “4”, “5”, and “6”applied thereto correspond to the first-viewpoint image 60 a, the secondviewpoint image 60 b, the third-viewpoint image 60 c, thefourth-viewpoint image 60 d, the fifth-viewpoint image 60 e, and thesixth-viewpoint image 60 f, respectively.

Such typical 6-viewpoint stereoscopic image display panel is effectivein some cases where a great number of people observe the images.However, such device is required to be structured to achieve light-rayseparation in the spaces for 6-viewpoints at all times. Thus, comparedto the case of the 2-viewpoint stereoscopic image display panel, the 3Dresolution is decreased to ⅓ in the lateral direction. Therefore, whenthe 3D resolution in the vertical direction is adjusted in accordancewith the decrease of the resolution in the lateral direction, theresolution as a whole is decreased to 1/9. This deteriorates thestereoscopic image quality greatly.

In this regard, the stereoscopic image display device 1 according to thefirst exemplary embodiment described by referring to FIG. 7 to FIG. 11has the function for generating the images for 6-viewpoints and formsthe stereoscopic regions as appropriate according to the position of theobserver 100 by using each of the images, so that it is possible to givemotion parallax effectively without decreasing the resolution unlike theabove-described case.

While the cases of the stereoscopic image display using the 2-viewpointstereoscopic image display panel and the 6-viewpoint image data aredescribed above, the structure of the first exemplary embodiment is notlimited to that. That is, it is also possible to employ the structureusing a combination of the 2-viewpoint stereoscopic image display paneland 10-viewpoint image data, a combination of a 4-viewpoint stereoscopicimage display panel and 12-viewpoint image data, or the like. With thecases of such structures, the effects same as those described above canbe acquired as well.

Therefore, the structural content according to the first exemplaryembodiment can be generally expressed as follows when the combinationsof the stereoscopic image display panel and the image data are takeninto consideration and the number of viewpoints of the stereoscopicimage display panel is defined as N (N is a natural number of 2 orlarger).

When the number of image data of each viewpoint to be inputted to thestereoscopic image display panel is defined as J in the stereoscopicimage display panel that is structured so that the main lobe exists inthe front side towards the display surface, it is desirable to satisfy arelation of “J>N” and, in addition, to satisfy a relation of “J=s×N (sis an integer of 2 or larger)” Note here that the front side towards thedisplay surface means the direction of the observer 100 when theobserver 100 is located on a normal line (referred to as a displaynormal line hereinafter) from the display surface having the center ofthe image display as the start point.

As in FIG. 5 which shows a state where the center line of the main lobeand the display normal line are almost consistent, it is so designedthat those are almost consistent with each other in most of the cases ofthe even-number viewpoints (N=2, 4, 6, - - - ).

In the meantime, with the cases of the odd-number viewpoints (firstneighboring viewpoint region: N=3, 5, 7, - - - ), the center line of themain lobe may be consistent with the display normal line or the centerline may be shifted with respect to the display normal line at an anglecorresponding to the viewpoint pitch or angles smaller than that.

In the latter case, when the angle formed roughly between the centerline of the main lobe and the display normal line is within a prescribedangle (within an angle of a tan (e/OD) provided that the viewpoint pitchat the optimum viewing distance OD is e, it is desirable to set thevalue of the coefficient s as an odd number of 3 or larger in view ofthe symmetry between the main lobe and the side lobes. This makes itpossible to have the side lobes appeared almost bilaterally symmetricwith each other.

Meanwhile, when the angle formed between the center line of the mainlobe and the display normal line is equal to or larger than theprescribed angle, the value of the coefficient s may simply be set as 2or larger. It is not necessary to set any specific limitation.

The value of the coefficient s may be stored in advance to the memory 33shown in FIG. 1 or a prescribed value may be loaded as the coefficient svia the external IF 34.

In any case, the image generation processing unit 30 is structured togenerate images of J (=s×N) viewpoints based on the value of thecoefficient s according to the viewpoint number N of the stereoscopicimage display panel 10.

Further, it is also possible to provide an angle measuring function,which measures the angle formed between the center line of the main lobeand the display normal line based on the positional information, to therelative position calculating unit 40 or the image generation processingunit 30 and further to provide a comparing/setting function, whichcompares the formed angle with a tan (e/OD) and sets the coefficient sas an odd number of 3 or larger when the angle is equal to or smallerthan a tan (e/CD), to the image generation processing unit 30. Thismakes it possible to have the side lobes appeared almost bilaterallysymmetric with each other under a specific condition.

For generating the images, there are various methods such as a methodwith which cameras for the number of viewpoints are disposed virtuallyfrom the three-dimensional polygon data stored in the data accumulationunit 32 and real-time rendering is performed by the arithmeticcalculator 31, a method with which images for the number of viewpointsare generated from CZ data, etc.

When generating the images, a preset method may be used by the imagegeneration processing unit 30 among each of those kinds of methods or amethod may be selected as appropriate according to the use environmentand external signals. Further, it is also possible to employ a structurewith which images for J-viewpoints are generated in advance by the imagegeneration processing unit 30 and those are saved in the dataaccumulation unit 32.

This makes it possible to give motion parallax as multi-viewpoints whilemaintaining the high resolution with smaller number of viewpoints.

Further, in a case where the viewpoint number N is 3 or larger, theimage data to be outputted to the target viewpoint region may be updatedaccording to the moving direction of the observer 100.

Now, FIG. 13 shows the relation between the light-ray separation stateand the position of the observer when a 3-viewpoint (the viewpointnumber N=3) stereoscopic image display panel is used as the stereoscopicimage display panel 10. In FIG. 13, the viewpoint regions 50 a, 51 a, 52a as the first-viewpoint regions, the viewpoint regions 50 b, 51 b, 52 bas the second-viewpoint regions, and the viewpoint regions 50 c, 51 c,52 c as the third-viewpoint regions are shown as the viewpoint regionsformed by corresponding to the position of the observer, and a position110 a and a position 110 b are shown as the positions of the observer.

Further, FIG. 14 shows an example of the image data inputted to each ofthe viewpoint regions via the 3-viewpoint stereoscopic image displaypanel 10 in FIG. 13. In FIG. 14, B1, B2, B3, B4, B5, B6, B7, B8,B9, - - - are shown as the image data for each of the viewpoints, andthe size of faces of “5” and size of the faces of “3” of the dices aredifferent for each of the viewpoint images as shown in FIGS. 3 and 7.That is, as shown in FIG. 14, employed is the structure with which theface of “5” becomes gradually smaller and the face of “3” becomesgradually larger from B1 towards B9.

The method for generating the images shown in FIG. 14 is basically thesame as the method described by referring to FIG. 8. That is, thecameras for each of the viewpoints according to the number of image dataare disposed substantially in parallel to the x-axis, and a 3D object iscaptured with those to generate the images.

Further, FIG. 15 and FIG. 16 show the relation between the viewpointregions where the left and right eyes of the observer are located andthe parallax image data inputted to the first-, second-, andthird-viewpoint regions regarding two cases where the observer movesfrom the position 110 a to the position 110 d (FIG. 13: ←) and where theobserver moves from the position 110 d to the position 110 a (FIG. 13:→).

FIG. 15 is a table showing a case where the observer moves from theposition 110 a towards the minus side (left side) of the x-axisdirection.

First, a state where the observer is at the position 110 a is defined as(A). In the state of (A), the viewpoint region where the left eye of theobserver is located is 50 b, and the viewpoint region where the righteye is located is 50 c. Regarding the image data, the image data B6 isinputted to the viewpoint region 50 b, and the image data B7 is inputtedto the viewpoint region 50 c. The image data B5 or B8 is inputted to theviewpoint region 50 a where neither the left eye nor the right eye islocated.

Next, a state where the left eye of the observer is located in theviewpoint region 50 a and the right eye is located in the viewpointregion 50 b, respectively, is defined as (B), and it is assumed that theobserver moves from the state of (A) to the state of (B). In the stateof (B), the image data B5 is inputted to the viewpoint region 50 a, andthe image data B6 is inputted to the viewpoint region 50 b. Although notshown in FIG. 15, if the observer moves from the state of (A) to theplus side (right side) of the x-axis direction and the viewpoint regionwhere the right eye of the observer is located becomes 52 a, the imagedata B8 is inputted to the viewpoint region 52 a.

In the state of (B), it seems that there is naturally no problem even ifinput of the image data B7 is maintained in the viewpoint region 51 cwhere neither the left eye nor the right eye is located.

However, in the first exemplary embodiment, the display controller 24employs a structure with which the image data for the viewpoint region51 c where the left eye of the observer may possibly be located by themove of the observer towards the direction continued from the move from(A) to (B) (the left side) is switched in advance to the image data B4that corresponds to the state of (C).

More specifically, the image generation processing unit 30 has an imagedata switching processing function (not shown) which estimates a nextmoving position of the observer based on the moving direction of theobserver shown in the chronological positional information from therelative position calculating unit 40 and the information from thesensor (not shown) within the observer position measuring unit 45, andoutputs the image data (viewpoint images) corresponding to the estimatedposition to the display panel driving circuit 21.

That is, in a case where the stereoscopic image display panel 10separates the parallax image towards three or more viewpoints (N≧3), theimage generation processing unit 30 uses the image data switchingprocessing function to select the viewpoint image for the viewpointregion where neither the left eye nor the right eye of the observer islocated according to the moving direction of the observer and outputs ittowards the stereoscopic image display panel 10. In the state of (B),for example, the viewpoint region in a relation such as the viewpointregion 51 c with respect to the viewpoint region 50 a is referred to asa neighboring viewpoint region.

As in the case described above, when the observer moves from the stateof (B) to the state of (C) (in a state where the left eye of theobserver is located in the viewpoint region 51 c and the right eye islocated in the viewpoint region 50 a, respectively), a state where theimage data B4 is inputted to the viewpoint region 51 c where the lefteye of the observer is located is maintained in the state of (C) asshown in FIG. 15.

Further, by taking the moving direction form (B) to (C), the displaycontroller 24 employs a structure with which the image data for theviewpoint region 51 b where the left eye of the observer may possibly belocated by the continuous move of the observer towards the samedirection is switched in advance to the image data B3 that correspondsto the state of (D).

The display controller 24 also repeats those processing actions in thefollowing states of (D) and (E). Through employing such structure, itbecomes possible to present stereoscopic images with an extremely smallamount of uncomfortableness felt even in a case where the moving speedof the observer is high.

In FIG. 15, each image data with an underline is the image data on whichthe switching processing is performed based on the estimation result ofthe moving position of the observer.

FIG. 16 shows a case where the observer moves from the position 110 dtowards the plus side (right side) of the x-axis direction. Further, ina state of (E) where the observer is located at the position 110 d, theimage data B3 is inputted to the viewpoint region 51 b where the righteye of the observer is located.

Considering the moving direction from (E) to (D) (right side), the statewhere the right eye of the observer is located in the first region bythe continuous move in the same direction is the state of (C). Thus, inthe first exemplary embodiment, it is structured to switch in advancethe image data for the first-viewpoint region (the viewpoint region 52 aand the like) to the image data B5 that corresponds to the state of (C)when the observer moves from the state of (E) to the state of (D).

When the observer moves from the state of (D) to the state of (C), astate where the image data B5 is inputted to the viewpoint region 50 awhere the right eye of the observer is located is maintained and thatstate is maintained also in the state of (C).

Considering the moving direction from (D) to (C) (right side), the statewhere the right eye of the observer is located in the second-viewpointregion by the continuous move in the same direction is the state of (B).Thus, as in the case described above, it is structured to switch inadvance the image data for the second-viewpoint region (the viewpointregion 50 b and the like) to the image data B6 that corresponds to thestate of (B).

Similarly, in the state of (B), by considering the moving direction(right side) from (C) to (B), it is also structured to switch in advancethe image data for the viewpoint region 50 c, in which the right eye ofthe observer may possibly be located due to the continuous move towardsthe same direction, to the image data B7 that corresponds to the stateof (A).

That is, the display controller 24 also repeats those processing actionsin the following states of (B) and (A). Through employing suchstructure, it becomes possible to present stereoscopic images with anextremely small amount of uncomfortableness felt even in a case wherethe moving speed of the observer is high.

In FIG. 16, each image data with an underline is also the image data onwhich the switching processing is performed based on the estimationresult of the moving position of the observer.

Incidentally, the time required for the processing executed by theobserver position measuring unit 45 and the relative positioncalculating unit 40 described by referring to FIG. 1 and the like isideally desired to be close to 0 as much as possible. However, inpractice, it takes 0.1 second or more for data transfer, calculation,and the like.

In such case, when outputting the image data from the image generationprocessing unit 30 based on the signals (positional information) fromthe relative position calculating unit 40, there is a possibility thatthe observer is not remaining at the measured position. Particularlywhen the moving speed of the observer is high, the left and right eyesof the observer may be located out of the estimated viewpoint regions.Thus, in worst cases, a prescribed stereoscopic image may not be sensed.

In view of such issue, the first exemplary embodiment employs thestructure with which the image data selected in advance by taking themoving direction of the observer into consideration is inputted inadvance for the target viewpoint region as described above. Therefore,it is possible to give smooth motion parallax for the observer who ismoving at a high speed without deteriorating the stereoscopicvisibility,

That is, as clearly shown in FIG. 15 and FIG. 16 in which an underlineis applied to the image data, significant data according to the movingdirection of the observer is inputted prior to the move of the observer.This makes it possible to present stereoscopic images without a sense ofuncomfortableness even when the moving speed of the observer is high.

Then, FIG. 17 shows an example of the relation between the light-rayseparation state and the observer when a 4-viewpoint (viewpoint numberN=4) stereoscopic image display panel is employed as the stereoscopicimage display panel 10. In FIG. 17, the viewpoint regions 50 a, 51 a, 52a as the first-viewpoint regions, the viewpoint regions 50 b, 51 b, 52 bas the second-viewpoint regions, the viewpoint regions 50 c, 51 c, 52 cas the third-viewpoint regions, the viewpoint regions 50 d, 51 d, 52 das the fourth-viewpoint regions are shown as the viewpoint regionsformed by corresponding to the position of the observer, and theposition 111 a and the position 111 b are shown as the positions of theobserver.

Further, the parallax image data inputted to each of the viewpointregions via the stereoscopic image display panel 10 will be described byreferring to FIG. 14 as in the case of FIG. 13.

FIG. 18 and FIG. 19 show the relation between the viewpoint regionswhere the left and right eyes of the observer are located and theparallax image data inputted to the first-, second-, third-, andfourth-viewpoint regions regarding two cases where the observer movesfrom the position 111 a to the position 111 d (FIG. 17: ←) and where theobserver moves from the position 111 d to the position 111 a (FIG. 17:→).

FIG. 18 is a table showing a case where the observer moves from theposition 111 a shown in FIG. 17 towards the minus side (left side) ofthe x-axis direction.

First, a state where the observer is at the position 111 a is defined as(A). In the state of (A), the viewpoint region where the left eye of theobserver is located is 50 b, and the viewpoint region where the righteye is located is 50 c. Regarding the image data, the image data B6 isinputted to the viewpoint region 50 b, and the image data B7 is inputtedto the viewpoint region 50 c. The image data B5 is inputted to theviewpoint region 50 a where neither the left eye nor the right eye islocated, and the image data B8 is inputted to the viewpoint region 50 d,respectively.

Even in a case where the observer moves from (A) to (B), the image dataB5 is continuously inputted to the viewpoint region 50 a. That is, as inthe case of the state of (A), the image data B5 is inputted to theviewpoint region 50 a where the left eye of the observer is located inthe state of (B).

Further, when the observer moves from (A) to (B), it seems that there isno problem even if input of the image data B8 and B7 is maintained forthe viewpoint regions 51 d and 51 c where neither the left eye nor theright eye is located. However, as in the case of the 3-viewpointstereoscopic image display panel described above (FIG. 13), employed isa structure with which, by taking the moving direction (left side) from(A) to (B) into consideration, the image data for the viewpoint region51 d where the left eye of the observer may possibly be located by thecontinuous move of the observer towards the same direction is switchedin advance to the image data B4 that corresponds to the state of (C).

In addition, in the stereoscopic image display device 1 according to thefirst exemplary embodiment which employs the 4-viewpoint stereoscopicimage display panel, the display controller 24 selects and switches theimage data for the viewpoint region 51 c where the left eye of theobserver may possibly be located due to the continuous move made furtherin the same direction according to the extent of the moving speed v1from (A) to (B) at the same time as the above-described switchingprocessing.

In the state of (B), for example, the viewpoint regions in a relationsuch as the viewpoint regions 51 d, 51 c with respect to the viewpointregions 50 a are referred to as a first neighboring viewpoint region anda second neighboring viewpoint region, respectively.

More specifically, the image generation processing unit 30 has an imagedata selection processing function (not shown) which generates inadvance image data for the viewpoint region where the eyes of theobserver may possibly be located due to the continuous move made furtherbased on the chronological positional information from the relativeposition calculating unit 40 and the information regarding the movingspeed of the observer from the sensor (not shown) within the observerposition measuring unit 45, and outputs the image data (viewpointimages) corresponding to the estimated position to the display paneldriving circuit 21.

That is, in a case where the stereoscopic image display panel 10separates the parallax image towards four or more viewpoints (N≧4), theimage generation processing unit 30 uses the image data selectionprocessing function to select the parallax image for the viewpointregion where neither the left eye nor the right eye of the observer islocated according to the moving direction and the moving speed of theobserver, and outputs it towards the stereoscopic image display panel10.

The image data selection processing function is a function which selectsthe image data by comparing the moving speed v1 with a threshold valuevth set in advance. More specifically, it is structured to select theimage data B7 when the moving speed v1 is equal to or smaller than thethreshold value vth (v1≦vth), and to select the image data B3 when themoving speed v1 is larger than the threshold value vth (v1>vth).

That is, the image data B7 is maintained when the moving speed v1 isequal to or smaller than the threshold value vth, and the image data isswitched to B3 when the moving speed v1 is larger than the thresholdvalue vth.

As shown in FIG. 18 by applying underlines, each of the structuralmembers within the display controller 24 is structured to functioneffectively to execute the same processing actions even when theobserver moves from (B) to (C) and further to (D) and (E). This makes itpossible to present the stereoscopic images giving no sense ofuncomfortableness, which corresponds to the observer whose moving speedis high.

FIG. 19 is a table showing a case where the observer moves from theposition 111 d shown in FIG. 17 towards the plus side (right side) ofthe x-axis direction.

First, a state where the observer is at the position 111 d is defined as(E). In the state of (E), the viewpoint region where the left eye of theobserver is located is 51 b, and the viewpoint region where the righteye is located is 51 c. Regarding the image data, the image data B2 isinputted to the viewpoint region 51 b, and the image data B3 is inputtedto the viewpoint region 51 c. The image data B1 and B4 are inputted,respectively, to the viewpoint regions 51 a and 51 d where neither theleft eye nor the right eye is located.

When the observer moves from (E) to (D), the image data B5 iscontinuously inputted from the state of (E) to the viewpoint region 51 dwhere the right eye is located.

At the time of this move, as in the case described by referring to FIG.18, employed is a structure with which, by taking the moving direction(right side) from (E) to (D) into consideration, the image data for theviewpoint region 50 a where the left eye of the observer may possibly belocated by the continuous move of the observer towards the samedirection is switched in advance to the image data B5 that correspondsto the state of (C) and, at the same time, the image data for theviewpoint region 51 b where the left eye of the observer may possibly belocated by the continuous move of the observer made further towards thesame direction is selected and switched according to the extent of amoving speed v2 from (E) to (D) (the image data selection processingfunction).

That is, as in the case described above, the display controller 24 isstructured to maintain the image data B2 when the moving speed v2 isequal to or smaller than the threshold value vth (v2≦vth) and to selectand switch to the image data B6 when the moving speed v2 is larger thanthe threshold value vth (v2>vth) by the image data selection processingfunction (not shown) provided within the image generation processingunit 30.

As shown in FIG. 19 by applying underlines, each of the structuralmembers within the display controller 24 is structured to functioneffectively to execute the same processing actions even when theobserver moves from the state of (D) to the state of (C) and furthermoves to (B) and (A). This makes it possible to present the stereoscopicimages giving no sense of uncomfortableness, which corresponds to theobserver whose moving speed is high.

Looking into FIG. 18 and FIG. 19, it can be found that the image datawith underlines (viewpoint regions regarding the underlined image data)vary depending on the directions to which the observer moves. This showsthat the stereoscopic image display device 1 of the first exemplaryembodiment can present flexible image data according to the movingdirection and the moving speed of the observer.

As described, through employing the structure with which correspondingimage data is inputted in advance not only for the next viewpoint region(the first neighboring viewpoint region) where the left eye of theobserver moves but also for the next but one viewpoint region (thesecond neighboring viewpoint region) where the observer may possibly belocated by the farther move in a case where the moving speed of theobserver is high, it becomes possible to present the stereoscopic imageswith a sense of a still smaller amount of uncomfortableness.

As described above by referring to FIG. 17 and the like, the processingaction for selecting the image data according to the moving directionand the moving speed of the observer is the structural content appliedwhen the viewpoint number N is 4 or larger. The target viewpoint regions(selected input target regions) when selecting and inputting image databy the image data selection processing function (not shown) are thefirst neighboring viewpoint, the second neighboring viewpoint, - - - ,and the (N-2)-th neighboring viewpoint by having the viewpoint region ofeither the left or right eye as the moving direction is taken as thestart point.

That is, in a case where the viewpoint number N is 4 or larger (N≧4),the image generation processing unit 30 selects the first to the(N-2)-th neighboring viewpoints by having, as the start point, theviewpoint region where the observer is located right before the move asthe viewpoint region where neither the left nor the right eye of theobserver is located by using the image data selection processingfunction.

Further, while the cases of using a single piece of image data for eachviewpoint region is used for explanations presented heretofore, it isalso possible to use two or more pieces of image data for each viewpointregion.

As an example thereof, a case of using four pieces of image data foreach viewpoint region for a 2-viewpoint stereoscopic image display panelwill be described.

Here, a case where the position of the observer 100 is at FIG. 9C willbe described first by referring to FIG. 6, FIG. 9, FIG. 14, and FIG. 10that is an enlarged view of FIG. 9C.

FIG. 10 that is an enlarged view of FIG. 9C shows a state where thefirst-viewpoint region 51 a where the left eye is located is dividedinto four in the x direction to provide first-viewpoint sub-regions 51aa, 51 ab, 51 ac, and 51 ad.

In accordance with the positions where the left eye of the observer 100moves from −x towards +x direction (plus direction of the x-axis), onepiece of image data selected from the image data B1, B2, B3, and B4shown in FIG. 14 is displayed for each of the first-viewpointsub-regions. Specifically, the image data B1 is displayed when the lefteye of the observer 100 is located in 51 aa, the image data B2 isdisplayed when the left eye is located in 51 ab, the image data B3 isdisplayed when the left eye is located in 51 ac, and the image data B4is displayed when the left eye is located in 51 ad.

Similarly, the second-viewpoint region where the right eye is located isdivided into four in the x-axis direction (not shown), and one piece ofimage data selected from the image data B5, B6, B7, and B8 shown in FIG.14 is displayed for each of the second-viewpoint sub-regions (not shown)in accordance with the positions where the right eye of the observer 100moves from −x towards +x direction.

Those image data B1 to B8 are generated by the image generationprocessing unit 30 shown in FIG. 1, and the image generation processingunit 30 output those to the display panel driving circuit 21 as the 3Ddata.

Then, when the observer 100 moves to the position shown in FIG. 9B, asin the case of FIG. 9C described above, the image data B5, B6, B7, andB8 shown in FIG. 14 are displayed for each of the sub-regions (notshown) of the second-viewpoint region 51 b where the left eye is locatedand image data B9, B10, B11, and B12 shown in FIG. 14 (B10 andthereafter are not shown) are displayed for each of the sub-regions (notshown) of the first-viewpoint region 50 a where the right eye islocated, respectively.

Further, when the observer 100 moves to the position shown in FIG. 9A,as in the case of FIG. 9C described above, the image data B9, B10, B11,and B12 shown in FIG. 14 (B10 and thereafter are not shown) aredisplayed for each of the sub-regions (not shown) of the first-viewpointregion 50 a where the left eye is located and image data B13, B14, B15,and B16 (not shown) shown in FIG. 14 are displayed for each of thesub-regions (not shown) of the second-viewpoint region 50 b where theright eye is located, respectively.

As described, in a case where the four pieces of image data are used foreach viewpoint region, the four pieces of image data are treated as asingle piece of image data group (e.g., image data enclosed within arectangle with a broken line in FIG. 14). Thereby, it is possible toachieve the processing content in which the image data group is switchedaccording to the moving direction of the observer 100 when the viewpointnumber N is 3-viewpoints, and the image data group is switched accordingto the moving direction and the moving speed of the observer 100 whenthe viewpoint number N is 4-viewpoints or more. This makes it possibleto present stereoscopic images in which a sense of uncomfortablenesscaused due to the move of the observer 100 is greatly eased.

While the case of treating the four pieces of image data as a singlepiece of image data group is described herein, the stereoscopic imagedisplay device 1 according to the first exemplary embodiment is notlimited only to that. That is, it is also possible to employ a structurewith which each of the viewpoint regions for the first viewpoint tofourth viewpoint regions is divided into four is taken as the apparentviewpoint regions and the image data to be displayed as appropriate onthe apparent viewpoint regions are selected and switched according tothe moving direction and the moving speed of the observer 100. Thismakes it possible to present stereoscopic images in which a sense ofuncomfortableness is eased more for the move of the observer.

As the number of the image data for each viewpoint region is increased,extremely smooth motion parallax can be acquired. When the number ofimage data is increased, it is desirable to increase the capacity of thedata accumulation unit 32 and the memory 33 within the displaycontroller 24 accordingly to increase the speed of the processingcapacity of the arithmetic calculator 31.

For example, in a case where twenty pieces of image data are used foreach viewpoint region with the 2-viewpoint stereoscopic image displaypanel, it is possible to acquire motion parallax which corresponds to a120-viewpoint stereoscopic image display panel even when only the mainlobe and the side lobes on both sides thereof are considered.

Note here that 3D resolution of the 120-viewpoint stereoscopic imagedisplay panel generally becomes 1/60 of that of the 2-viewpointstereoscopic image display panel, so that the 3D image quality isdeteriorated extremely.

However, with the structure of the first exemplary embodiment, it ispossible to achieve both high 3D resolution and smooth motion parallaxas described above.

(Explanations of Operations)

Next, operation contents of the stereoscopic image display device 1described by referring to FIG. 1 to FIG. 19 will be described brieflybased on a flowchart shown in FIG. 20.

First, the observer position measuring unit 45 measures the position ofthe observer 100 by using a camera provided to itself, and outputs themeasurement result (FIG. 20: S101).

Then, the relative position calculating unit 40 calculates the relativeposition of the left eye 101 and the right eye 102 with respect to thereference point (the center of the image display herein) set in advanceon the stereoscopic image display panel 10 based on the measurementresult acquired from the observer position measuring unit 45, andoutputs the calculation result (positional information) (FIG. 20: S102).

Then, the image generation processing unit 30 generates and outputs 3Ddata and control signals based on the positional information acquiredfrom the relative position calculating unit 40 (FIG. 20: S103).

Here, the image generation processing unit 30 in a case where the3-viewpoint stereoscopic image display panel is employed estimates thenext moving position of the observer based on the information regardingthe moving direction of the observer, generates image data for theviewpoint region where the both eyes of the observer are not located bycorresponding to the estimated position, and outputs the image data tothe display panel driving circuit 21 (the image data switchingprocessing function) (FIG. 20: S103).Further, the image generation processing unit 30 in a case where the4-viewpoint stereoscopic image display panel is employed generates inadvance image data for the viewpoint region where the both eyes of theobserver are not located based on the information regarding the movingspeed of the observer, and outputs the image data to the display paneldriving circuit 21 (the image data switching processing function) (FIG.20: S103).

Subsequently, the display panel driving circuit 21 drives the displaypanel based on the 3D data acquired from the image generation processingunit 30, and the image distribution control circuit 22 controls theactions of the light-ray separating module 12 according to the controlsignals from the image generation processing unit 30 (FIG. 20: S104).

It is also possible to put the execution content of each of theabove-described steps S101 to S104 (FIG. 20) into programs and have aseries of each of the control programs achieved by a computer.

Effects and the Like of First Exemplary Embodiment

In the first exemplary embodiment, the structure for expanding thenumber of viewpoints determined according to the device structure isemployed. Thus, it is possible to provide motion parallax withoutdeteriorating the resolution.

Further, with the stereoscopic image display device which executes imagecontrol according to the moving direction and the moving speed of theobserver, it becomes possible to provide stereoscopic images with asmaller amount of uncomfortableness.

Furthermore, even when the stereoscopic image display panel of smallnumber of viewpoints is employed, the image switching processing using aplurality of viewpoint images can be executed as appropriate asdescribed above. Therefore, it is possible to project the stereoscopicimages of high resolution with smooth motion parallax.

Further, since the relative position calculating unit 40 calculates therelative position of each of the both eyes of the observer based on themeasurement result from the observer position measuring unit 45, and theimage generation processing unit 30 generates and outputs significantimages and control signals based on the calculation result, thestereoscopic image display device according to the first exemplaryembodiment can generate and display significant parallax images whichcorrespond to the position of the observer.

As an exemplary advantage according to the invention, the presentinvention makes it possible to provide in particular a stereoscopicimage display device and a terminal device, which generate and displaysignificant parallax images corresponding to the position of theobserver.

Second Exemplary Embodiment

A second exemplary embodiment of the image display device according tothe present invention will be described by referring to FIG. 21 to FIG.26. Same reference numerals are used for the structural members and thelike equivalent to those of the first exemplary embodiment describedabove.

First, FIG. 21 shows a state of light-ray separation done by a2-viewpoint stereoscopic image display panel 10 according to the secondexemplary embodiment. As in the case of FIG. 5 and the like describedabove, FIG. 21 is a sectional view of a zx plane from the frontdirection of the y-axis (see FIG. 1).

When it is defined that the distance between the both eyes of theobserver is d (d=65 mm in general) and the viewpoint pitch between thefist-viewpoint region and the second-viewpoint region neighboringthereto is e, it is generally considered to satisfy a relation of “d≦e”for securing a prescribed stereoscopic region in the 2-viewpointstereoscopic image display panel. Each of the members is structured onthe assumption of that.

However, as shown in FIG. 21, it is a feature of the 2-viewpointstereoscopic image display panel 10 herein that the relation between dthat is the distance between the both eyes of the observer and theviewpoint pitch e between the first-viewpoint region 50 a and theneighboring second-viewpoint region 50 b or 51 b satisfies “d>e”. Otherstructural contents are the same as the state of FIG. 9A of the firstexemplary embodiment described above.

For describing the viewpoint pitch in terms of the 3D crosstalk, the 3Dcrosstalk characteristic in the center part of the display surface isshown in FIG. 22. In this case, the lateral axis is the x-axis directionat the optimum viewing distance (OD: see FIG. 5), and the longitudinalaxis is the values of the 3D crosstalk (3D crosstalk values).

While the lateral axis is defined as the extent in the x-axis direction(distance in the x-axis direction) in the second exemplary embodiment,the lateral axis regarding the 3D crosstalk characteristic may bestructured to show the viewing angles calculated with a tangent of theextent in the x-axis with respect to the optimum viewing distance.

The 3D crosstalk is defined as a leakage and mixture amount of an imageof the other eye for each of the left and right eyes. Therefore, a stateof 100% crosstalk means a state where a left-eye image and a right-eyeimage are mixed in a ratio of 1:1. Further, the viewpoint pitch and thex-axis range between the points at which the 3D crosstalk becomes 100%are equivalent, so that those are shown with a same reference code “e”in FIG. 22.

When the value of the 3D crosstalk is increased, a sense of stereopsisis lost and sense of fatigue and other unpleasant influences may begiven to the observer. Thus, it is desirable to set the value of thecrosstalk (crosstalk amount) to be equal to or less than a referencevalue CT1 that is set in advance. In general, the reference value CT1 isset to be equal to or less than 10%. Further, when it is set to be equalto or less than 5%, the above-described issues can be lightened moreeffectively.

Further, out of a viewing field range defined by the reference valueCT1, FIG. 22 shows a part thereof by defining a low 3D crosstalk rangeas f and a high 3D crosstalk range as c. When the left and right eyes ofthe observer are within the low 3D crosstalk range f, the observer 100can visually recognize a fine stereoscopic image.

70 a and 72 a shown with a solid line correspond to the first-viewpointregions 50 a and 52 a shown in FIG. 21, respectively, and 70 b and 71 bshown with a broken line correspond to the second-viewpoint regions 50 band 51 b shown in FIG. 21, respectively.

When the value of the distance d between the both eyes exceeds the valueof the viewpoint pitch (e+f), the observer 100 cannot visually recognizea stereoscopic image, which means that the so-called stereoscopic regiondoes not exist.

Therefore, the second exemplary embodiment is structured to satisfy therelation of “(e+f)>d>e”.

FIG. 23 shows the viewpoint regions (including the crosstalk region)where the left and right eyes of the observer are located when theobserver moves from the state of (A) shown in FIG. 22 to the minus sideof the x-axis direction, i.e., when the observer moves from the state of(A) to each of the states shown in (B), (C), and (D) in this order.

Further, FIG. 24 shows a corresponding relation between the viewpointregions where the left and right eyes of the observer are located andthe image data of the first-viewpoint region and the second-viewpointregion. Here, shown is a case where the image data shown in FIG. 7 areused.

Further, as a comparative example, FIG. 25 and FIG. 26 show thecorresponding relation and the like as those described above in a caseof using the stereoscopic image display panel satisfying the relation of“d<e”.

“2”, “3”, and “4” shown in the section of the image data (withinparentheses) in FIG. 21, or FIG. 24 and FIG. 26 correspond to thesecond-viewpoint image 60 b, the third-viewpoint image 60 c, and thefourth-viewpoint image 60 d, respectively. Further, in the section ofthe viewpoint regions (within parentheses) in FIG. 24 and FIG. 26, shownare the ranges where each of the both eyes of the observer is located(low 3D crosstalk range: f, high 3D crosstalk range: c).

Comparing FIG. 23 with FIG. 25, the both are consistent in respect thatboth of the left and right eyes are in a state of entering the low 3Dcrosstalk region in a case where the position of the observer is (A) or(D) and that one of the left and right eyes enters in the low 3Dcrosstalk region and the other is in a state entering the high 3Dcrosstalk region in a case where the position of the observer is (B) or(C).

However, when the observer moves from (A) to (B), there is a differencein respect that the high 3D crosstalk region is generated from the lefteye in the case of “d>e” (FIG. 23) as in the second exemplary embodimentand that the high 3D crosstalk region is generated from the right eye inthe case of “d<e” (FIG. 25) as in the comparative example.

This is the same for the case where the observer moves from (B) to (C).

In the case of (B) or (C), the image data projected in thefirst-viewpoint region and the second-viewpoint region are determineddepending on which of the left and right eyes of the observer exists inthe low 3D crosstalk region f after the move based on the state beforethe observer moves. That is, in such state, the display controller 24 isstructured to select the image data by giving priority to the sense ofthe eye that is located in the low 3D crosstalk region f (an image datapriority selecting function).

For example, as shown in FIG. 24, in the case of (A), the left eyeexists in 70 a that is the first-viewpoint region and the right eyeexists in 70 b that is the second-viewpoint region. As the image data,60 c that is the third-viewpoint image is projected to thefirst-viewpoint region, and 60 d that is the fourth-viewpoint image isprojected to the second-viewpoint region.

When the observer moves from this state to the state of (B), the lefteye comes to exist in the high 3D crosstalk region c where thefirst-viewpoint region 70 a and the second-viewpoint region 71 b aremixed, and the right eye comes to exist in 70 b that is thesecond-viewpoint region. In that case, the priority is given to thesense of the right eye as described above, so that the displaycontroller 24 uses 60 d that is the fourth-viewpoint image same as thestate of (A) for the second-viewpoint region and uses thethird-viewpoint image 60 c same as the state of (A) also for thefirst-viewpoint region.

Then, when the observer moves from (B) to (C), the left eye comes toexist in 71 b that is the second-viewpoint region and the right eyecomes to exist in the high 3D crosstalk region c where thefirst-viewpoint region 70 a and the second-viewpoint region 70 b aremixed. In that case, as in the above-described case, the priority isgiven to the sense of the left eye as described above, so that thedisplay controller 24 switches the image used for the second-viewpointregion to the second-viewpoint image 60 b different from the state of(B), and continuously uses the third-viewpoint image 60 c for thefirst-viewpoint region.

The second-viewpoint image switching processing is the processing bytaking the move of the observer in the minus direction of the x-axisinto consideration. That is, it is the processing for forming astereoscopic region with the third-viewpoint image 60 c maintained inthe first-viewpoint region (for avoiding a reverse view state) when theobserver comes to be in the state of (D).

Thereby, the viewpoint image same as that of the state of (C) can beused as it is in both the first-viewpoint region and thesecond-viewpoint region, when the observer moves from (C) to (D) and theleft eye comes to exist in 71 b that is the second-viewpoint region andthe right eye comes to exist in the first-viewpoint region 70 a.

That is, the second exemplary embodiment is structured to switch theimage data only once in the second-viewpoint region while the observermoves from (A) to (D).

The case shown in FIG. 26 is the same. That is, when the observer movesfrom (A) to (B), the left eye comes to exist in 70 a that is thefirst-viewpoint region, and the right eye comes to exist in the high 3Dcrosstalk region c where the first-viewpoint region 70 a and thesecond-viewpoint region 71 b are mixed. Thus, the priority is given tothe sense of the left eye, so that the third-viewpoint image 60 c sameas the state of (A) is used for the first-viewpoint region and thefourth-viewpoint image 60 d same as the state of (A) is used also forthe second-viewpoint region.

Then, when the observer moves from (B) to (C), the left eye comes toexist in the high 3D crosstalk region c where the first-viewpoint region70 a and the second-viewpoint region 71 b are mixed, and the right eyecomes to exist in 70 a that is the first-viewpoint region. Thus, thepriority is given to the sense of the right eye, switching processingwhich takes the move of the observer in the minus direction of thex-axis into consideration is executed with which the image used for thesecond-viewpoint region is switched to the second-viewpoint image 60 band the third-viewpoint image 60 c is used as it is for thefirst-viewpoint region.

As described, in both of the cases satisfying “d>e” (the secondexemplary embodiment) and satisfying “d<e” (comparing example), imagedata is switched in one of the viewpoint regions when the observer movesfrom (B) to (C).

Incidentally, in a case of “d=e”, the states of (B) and (C) in FIG. 26are mixed. Thus, when the observer moves from the state of (A) to thatstate, both the left and right eyes come to exist in the high 3Dcrosstalk region and the state directly transits to the state of (D)when the observer further moves therefrom.

That is, in the state of (B)+(C), neither the left eye nor the right eyeenters the low 3D crosstalk region. Thus, an image with a sense ofuncomfortableness is inputted momentarily so that the use of thestereoscopic image display panel employing a structure satisfying “d=e”is not considered to be preferable.

As described, with the use of the structure of the second exemplaryembodiment, it is possible to execute the image switching processing inthe case of “d>e” in the same manner as in the case of “d<e”. As aresult, a still larger number of viewpoint regions can be provided whenthe extent of the visible range in the x-axis is constant, so thatsmooth motion parallax can be presented.

Effects and the Like of Second Exemplary Embodiment

In the second exemplary embodiment, the stereoscopic image display panelis structured in such a manner that the viewpoint pitch e between theneighboring viewpoint regions becomes smaller than the distance dbetween both eyes of the observer by taking the existence of the low 3Dcrosstalk range f and the high 3D crosstalk range c into account. Thatis, significant light-ray separation can be achieved herein by using asmall pitch, thereby making it possible to present smooth motionparallax even with 2-viewpoints.

Other structures and operations are the same as those of the firstexemplary embodiment described above, and other operating effects to begenerated thereby are also the same. In a case where two or more imagedata are used for each viewpoint region in particular, still smoothermotion parallax can be presented.

Third Exemplary Embodiment

A third exemplary embodiment of the image display device according tothe third exemplary embodiment will be described by referring to FIG. 27and FIG. 29.

It is the feature of the third exemplary embodiment to employ astructure of a stereoscopic image display device of 3-viewpoints or morein which a relation of “e×(N-2)≦d≦e×(N-1)” is satisfied, where thenumber of viewpoints is N (N is a natural number of 2 or larger), eachviewpoint pitch is e, and the distance between both eyes of the observeris d. Note here that same reference numerals are used for the structuralmembers and the like same as those of the first exemplary embodimentdescribed above.

First, FIG. 27 shows a state of light-ray separation done by a4-viewpoint stereoscopic image display panel 10 as an example of thethird exemplary embodiment of the present invention.

As shown in FIG. 27, on the display surface side of the stereoscopicimage display panel 10, there are regions for outputting image data for4-viewpoints, which are constituted with the first-viewpoint regions 50a, 51 a, 52 a, - - - , the second-viewpoint regions 50 b, 51 b, 52b, - - - , the third-viewpoint regions 50 c, 51 c, 52 c, - - - , and thefourth-viewpoint regions 50 d, 51 d, 52 d, - - - . Further, the mainlobe 90 is constituted with 50 a, 50 b, 50 c, and 50 d, the side lobe 91is constituted with 51 a, 51 b, 51 c, and 51 d, and the side lobe 92 isconstituted with 52 a, 52 b, 52 c, and 52 d, respectively.

A case where the left eye of the observer exists in the side lobe 91 andthe right eye exists in the main lobe 90 is a reverse view region as inthe case described in the first exemplary embodiment.

The above-described relation of “e×(N-2)≦d≦e×(N-1)” in the case of4-viewpoints (N=4) shown in FIG. 27 is a relation of “2 e≦d≦3e”.

Next, the relation between the positions of the observer and each of theviewpoint regions is shown in FIG. 28, and a case where the observermoves from the position 112 a to the position 112 d will be describedherein.

First, when the observer exists in the position 112 a, the left eye ofthe observer exists in the first-viewpoint region 50 a and the right eyeof the observer exists in the third-viewpoint region 50 c or thefourth-viewpoint region 50 d according to the relation of“e×(N-2)≦d≦e×(N-1)” described above.As the image data inputted to each viewpoint in this case, the imagedata shown in FIG. 14 that is referred in the first exemplary embodimentis employed.

FIG. 29 is a table which shows the corresponding relation between theviewpoint regions where the left and right eyes of the observer arelocated and the image data used for each of the viewpoint regions in acase where the observer moves from the position 112 a towards theposition 112 d successively.

As shown in FIG. 29, in a state of (A) where the observer is in theposition 112 a, the image B5 is used for the first-viewpoint region 50a, the image B6 is used for the second-viewpoint region 50 b, the imageB7 is used for the third-viewpoint region 50 c, the image B8 is used forthe fourth-viewpoint region.

Therefore, fine stereoscopic images can be visually recognized in thisstate.

Next, a case where the left eye of the observer moves to thefourth-viewpoint region 51 d of the side lobe 91 (state of (B) shown inFIG. 29) is investigated. In that state, the right eye of the observerexists in the second-viewpoint region 50 b or the third-viewpoint region50 c as in the above-described case. This corresponds to the reverseview region described above. Thus, the third exemplary embodimentemploys a structure with which the display controller 24 switches theimage used for the fourth-viewpoint region 51 d to B4 from B8 based onthe observer position measurement information. Through employing suchswitching processing, it becomes possible to provide fine stereoscopicimages also for the observer located in that place.

Similarly, a case where the left eye of the observer moves to thethird-viewpoint region 51 c of the side lobe 91 (state of (C) shown inFIG. 29) is investigated. In that state, the right eye of the observerexists in the first-viewpoint region 50 a or the second-viewpoint region50 b. This corresponds to the reverse view region described above. Thus,the display controller 24 is structured to switch the image used for thethird-viewpoint region 51 c to B3 from B7 based on the observer positionmeasurement information. Through employing such switching processing, itbecomes possible to provide fine stereoscopic images also for theobserver located in that place.

Similarly, a case where the left eye of the observer moves to thesecond-viewpoint region 51 b of the side lobe 91 (state of (D) shown inFIG. 29) is investigated. In that state, the right eye of the observerexists in the fourth-viewpoint region 51 d or the first-viewpoint region50 a. This corresponds to the side lobe 91 or the reverse view. In suchcase, the display controller 24 is structured to switch the image usedfor the second-viewpoint region 51 b to B2 from B6 based on the observerposition measurement information for allowing the observer to visuallyrecognize finer stereoscopic images.

As described, through employing the structure which executes the imageswitching processing based on the observer position measurementinformation, it becomes possible to acquire smooth motion parallax bysuppressing deterioration of the resolution even with a multi-viewpointtype stereoscopic image display device of small number of viewpoints.

Effects and the Like of Third Exemplary Embodiment

As described above, with the stereoscopic image display device accordingto the third exemplary embodiment, it becomes possible to acquire smoothmotion parallax by suppressing deterioration of the resolution even witha multi-viewpoint type stereoscopic image display device of small numberof viewpoints.

Other structures and operations are the same as those of the firstexemplary embodiment described above, and other operating effects to begenerated thereby are also the same. In a case where two or more imagedata are used for each viewpoint region in particular, smooth motionparallax can be presented.

Fourth Exemplary Embodiment

A fourth exemplary embodiment of the image display device will bedescribed by referring to FIG. 30 to FIG. 43. In the first to thirdexemplary embodiments described above, the case where an image isoptically separated for a single observing direction to project parallaximages to the observer has been presented. However, in the fourthexemplary embodiment, a structure with which an image is opticallyseparated for two observing directions to project parallax images to theobserver is employed.

Note here that same reference numerals are used for the structuralmembers and the like same as those of the first exemplary embodimentdescribed above.

(Overall Structures)

FIG. 30 shows an example of a block diagram of a stereoscopic imagedisplay panel 210 according to the fourth exemplary embodiment.

The stereoscopic image display panel 210 includes: a display panel 211in which pixels (not shown) are arranged in matrix; and a light-rayseparating module 250 provided by being corresponded to each pixel onthe display surface side of the display panel 211.

The stereoscopic image display panel 210 can provide stereoscopic imagedisplay in each of two states which are a first layout state where thex-axis direction is the horizontal direction and a second layout statewhere the y-axis direction is the horizontal direction. Further, thehorizontal direction herein is defined as a substantially paralleldirection with a straight line connecting between the left eye and theright eye of the observer, and it is the same in the explanationshereinafter.

Specific examples of a display controller 25, the stereoscopic imagedisplay panel 210, the display panel 211, and the light-separatingmodule 250 shown in FIG. 42 to be described later are the same as thestructural contents described in the first exemplary embodiment (thedisplay controller 24, the stereoscopic image display panel 10, thedisplay panel 11, and the light-ray separating module 12).

FIG. 31 shows an example of the layout relation regarding pixels 221provided on the display panel and lens array elements 251 whichconstitute the light-ray separating module (lens array) 250.

As shown in FIG. 31A, the pixel 221 formed with 4×4 sub-pixels isconstituted with four primary colors of a red sub-pixel 234, a bluesub-pixel 235, a green sub-pixel 236, and a white sub-pixel 237.

Further, the 4×4 sub-pixels provided in a single pixel 221 are arrangedsuch that each of the primary colors does not overlap in both the x-axisdirection and the y-axis direction.

Here, R1, R2, R3, R4 as the red sub-pixels 234, B1, B2, B3, B4 as theblue sub-pixels 235, G1, G2, G3, G4 as the green sub-pixels 236, and W1,W2, W3, W4 as the white sub-pixels 237 are arranged as in FIG. 31A sothat each of the primary colors is not disposed to overlap with eachother.

FIG. 31B shows a state where the pixels 221 shown in FIG. 31A arearranged in matrix. Each of the lens array elements 251 constituting thelens array 250 is disposed at a position corresponding to each pixel221. Therefore, the stereoscopic image display panel 210 can distributethe light into the directions of 4-viewpoints both in the x-axisdirection and the y-axis direction.

The pixels 221 and the lens array elements 251 are arranged in parallelwith respect to the x-axis direction and the y-axis direction and at asame pitch, respectively.

The structures of the sub-pixels and the pixels according to the fourthexemplary embodiment shown in FIG. 31 can generally be expressed asfollows by using the natural number M and N (natural number of 2 orlarger) that is a multiple of M.

When sub-pixels constituted with M primary colors, the number ofsub-pixels constituting each pixel is a square of N(N-viewpoints×N-viewpoints), and the remainder when N is divided by M is0. Further, within each pixel, neighboring pixels for the twolight-separating directions are not of a same color. Furthermore, withineach pixel, the existence probability of the same color for the twolight-separating directions is the same for all the M primary colors.

In addition, the layout pitches of the sub-pixels for the twolight-separating directions are equivalent.

While a case where a single pixel is constituted with 4×4 sub-pixels isshown in FIG. 31. However, the structure is not limited only to that.For example, a display panel in which pixels each constituted with 3×3sub-pixels are arranged in matrix, etc., can also be applied as well.Further, while primary colors of RGBW (red, green, blue, white) are usedto describe the 4×4 sub-pixels, the colors are not limited only tothose. For example, instead of RGBW, RGBY (red, green, blue, yellow),CMYW (cyan, magenta, yellow, white), or the like may be employed aswell. Other than those, it is also possible to use fluorescent colors,pearl colors, interference colors as the primary colors.

In the structure of the 4×4 sub-pixels constituted with four primarycolors shown in FIG. 31A, the image data shown in FIG. 14 referred inthe first exemplary embodiment described above can be used.

Next, the relation between the input image and the sub-pixels will bedescribed by referring to FIG. 32 which shows the sub-pixel structure inthe first layout state.

Herein, the pixel 221 shown in FIG. 31 is functioned as a structurewhich is constituted with: a sub-pixel group 241 where sub-pixels W4,G4, B4, and R4 are disposed; a sub-pixel group 242 where sub-pixels B3,R3, G3, and W3 are disposed; a sub-pixel group 243 where sub-pixels G2,W2, R2, and B2 are disposed; and a sub-pixel group 244 where sub-pixelsR1, B1, W1, and G1 are disposed.

That is, in a case of using the input images as shown in FIG. 14, first,signals corresponding to the fifth-viewpoint image B5 are inputted tothe sub-pixel group 241, signals corresponding to the sixth-viewpointimage B6 are inputted to the sub-pixel group 242, signals correspondingto the seventh-viewpoint image B7 are inputted to the sub-pixel group243, and signals corresponding to the eighth-viewpoint image B8 areinputted to the sub-pixel group 244, respectively.

At that time, it is possible to acquire such an effect that there is nodifference generated in the number of display colors among each of theviewpoint images within a pixel since all of the sub-pixel groups 241 to244 are constituted with the four primary colors.

Subsequently, the relation between each of the viewpoint images and thesub-pixels will be described by referring to FIG. 33 regarding thesub-pixel structure in the second layout state that is rotated clockwiseby 90 degrees from the first layout state shown in FIG. 32. In FIG. 33,as in the case of FIG. 32, signals corresponding to the fifth-viewpointimage B5 are inputted to the sub-pixel group 245, signals correspondingto the sixth-viewpoint image B6 are inputted to the sub-pixel group 246,signals corresponding to the seventh-viewpoint image B7 are inputted tothe sub-pixel group 247, and signals corresponding to theeighth-viewpoint image B8 are inputted to the sub-pixel group 248,respectively.

As described, the same numbers of parallax images (viewpoint images) aregenerated in the x-axis direction and the y-axis direction within eachpixel, and the number of different colors of sub-pixels is equivalentwithin each viewpoint image. Thus, significant stereopsis can beachieved equivalently both in the first layout state and the secondlayout state.

Further, with such structure, the 3D resolution can be made equivalentfor the first layout state and the second layout state regardless of thenumber of viewpoints.

Further, since the viewpoint number N is a multiple of M colors as thegenerally expressed above so that generation of color moire can besuppressed. Note here that “color moiré” is unevenness in each colorsensed when the observer shakes the viewing field angles in the layoutdirections. When “color moire” occurs, the display quality isdeteriorated greatly.

As the light-ray separating module 250, static optical elements andactive optical elements can both be used. However, it is preferable touse active optical elements such as liquid crystal lens elements orliquid crystal barrier elements capable of separating light rays in thex-axis direction and the y-axis direction.

Incidentally, as shown in FIG. 34, through employing a structure withwhich each of the lens array elements 251 executes light-ray separationonly in the x-axis direction (each direction on the zx plane) that isthe light-ray separating direction and does not execute light-rayseparation in the y-axis direction (each direction on the yz plane)orthogonal thereto for the structural example of the sub-pixel groupsshown in FIG. 32, it is possible to avoid color cracking withoutdepending on the separating angles. The state shown in FIG. 34corresponds to the first layout state described above.

Each of the regions light-ray separated in the x-axis direction is asfollows. That is, 50 a, 51 a, and 52 a (not shown) are thefirst-viewpoint region corresponding to the sub-pixel group 241, 50 b,51 b (not shown) and 52 b are the second-viewpoint region correspondingto the sub-pixel group 242, 50 c, 51 c, and 52 c (not shown) are thethird-viewpoint region corresponding to the sub-pixel group 243, and 50d, 51 d, and 52 d (not shown) are the fourth-viewpoint regioncorresponding to the sub-pixel group 244.

In addition, the main lobe is constituted with 50 a, 50 b, 50 c, and 50d. A combination of 51 a, 51 b, 51 c, and 51 d or a combination of 52 a,52 b, 52 c, and 52 d correspond to the side lobe. The light rays 59 a,59 b, 59 c, and 59 d shown in the y-axis direction are not separated asshown in the drawing.

Further, also in the second layout state (corresponds to the structuralexample of the sub-pixel group shown in FIG. 33) in which the y-axisdirection is the horizontal direction, each of the lens array elements251 is structured in such a manner that light ray is separated only inthe y-axis direction but not in the x-axis direction.

Therefore, with the structure of the fourth exemplary embodiment, itbecomes possible to avoid color cracking without depending on theseparating angles on both of the layout states. Further, it is possibleto suppress such inconvenience that the color moire tends to becomeactualized as the separating angle is increased. That is, this structureis an effective means for improving the visibility.

Next, as the state where the observer moves along the x-axis directionfrom a state of being in a position 113 a shown in FIG. 34, the 3Dcrosstalk characteristic in the center part of the display surface ofthe stereoscopic image display panel 210 is shown in FIG. 35.

The details regarding the 3D crosstalk characteristic are the same asthe content described by referring to FIG. 23 and the like in the secondexemplary embodiment. Further, 70 a and 72 a correspond to thefirst-viewpoint regions 50 a and 52 a shown in FIG. 34, respectively, 70b corresponds to the second-viewpoint region 50 b shown in FIGS. 34, 70c and 71 c correspond to the third-viewpoint regions 50 c and 51 c shownin FIG. 34, respectively, and 70 d and 71 d correspond to thefourth-viewpoint regions 50 d and 51 d shown in FIG. 34, respectively.

In FIG. 35, the x-axis direction in the optimum viewing distance istaken on the lateral axis, and the values of the 3D crosstalk are takenon the longitudinal axis as in the case of FIG. 23. Similarly, thelateral axis regarding the 3D crosstalk characteristic may be shown asthe viewing field angles calculated by a tangent in the value of thex-axis with respect to the optimum viewing distance.

Here, FIG. 36 shows the corresponding relation between the viewpointregions where the left and right eyes of the observer are located andthe image data for the first-viewpoint region to the fourth-viewpointregion in a case where the observer moves to the minus direction of thex-axis successively as shown in FIG. 35A to FIG. 35D. The image datashown in FIG. 14 of the first exemplary embodiment described above areemployed as the image data to be used.

As shown in FIG. 36, when the observer moves from the position 113 b tothe position 113 c, i.e., when the observer enters the reverse viewregion from the main lobe, the display controller 24 switches the imagedata upon receiving the measurement result from the observer positionmeasuring unit 45 that has detected the observer position of that time.

The fourth exemplary embodiment employs the processing for switching allthe image data for the fourth-viewpoint regions when the observerreaches the state of (C). Thus, it is unnecessary to switch the imagedata until the observer enters the next reverse view region even whenthe observer moves further to the minus direction of the x-axis by goingthrough the state of (D).

When the observer moves from the position 113 a to the position 113 b or113 c to the position 113 d, the image data are not switched as shown inFIG. 36.

Further, it is also possible to structure each of the pixels arranged onthe display panel with a sub-pixel group shown in FIG. 37, and to allowthe observer to sense stereopsis by using those.

In FIG. 37A, the pixel 221 shown in FIG. 31 is constituted with: asub-pixel group 241′ where sub-pixels W4, B3, G4, and R3 are disposed; asub-pixel group 242′where sub-pixels G2, R1, W2, and B1 are disposed; asub-pixel group 243′ where sub-pixels B4, G3, R4, and W3 are disposed;and a sub-pixel group 244′ where sub-pixels R2, W1, B2, and G1 aredisposed. Similarly, in FIG. 37B, it is constituted with: a sub-pixelgroup 245′ where sub-pixels G2, R1, W2, and B1 are disposed; a sub-pixelgroup 246′ where sub-pixels R2, W1, B2, and G1 are disposed; a sub-pixelgroup 247′ where sub-pixels W4, B3, G4, and R3 are disposed; and asub-pixel group 248′ where sub-pixels B4, G3, R4, and W3 are disposed.

That is, FIG. 37A shows the sub-pixels within the pixel 221 in the firstlayout state and the corresponding lens element 251, and FIG. 37B showseach structure in the second layout state in which the y-axis directionis set to the horizontal direction by rotating the first layout stateclockwise by 90 degrees.

Further, FIG. 38 shows an example of the image information inputted toeach sub-pixel employed in the fourth exemplary embodiment.

In FIGS. 38, A1 to A9, B1 to B9, C1 to C9, and D1 to D9 (each of theimages lined laterally) are images of different horizontal parallax witheach other. For example, each of the images lined longitudinally such asA1, B1, C1, and D1 (images with different alphabets but same numericalnumber) are images of different vertical parallax with each other.

A significant feature of the image data shown in FIG. 38 is a differencein the size of the faces of “1” in the dices of the four viewpointimages (A to D) lined longitudinally. Note, however, that the size ofthe faces of “5” and size of the faces of “3” in the dices aredifferent, respectively, among each of the viewpoint images to which thenumbers 1 to 9 are applied, as in the case of FIG. 14 and the likedescribed above.

The method for generating the images shown in FIG. 38 is basically thesame as the method described by referring to FIG. 8. That is, each ofthe viewpoint cameras according to the number of image data are disposedsubstantially in parallel to the x-axis and the y-axis, and a 3D objectis captured with those to generate images.

Here, the relation between each of the viewpoint images and thesub-pixels in the first layout state will be described by referring toFIG. 37A and FIG. 38.

In a case of employing the input images shown in FIG. 38, for example,signals corresponding to the image B5 are inputted to the sub-pixelgroup 241′ of FIG. 37A, signals corresponding to the image B6 areinputted to the sub-pixel group 242′, signals corresponding to the imageC5 are inputted to the sub-pixel group 243′, and signals correspondingto the image C6 are inputted to the sub-pixel group 244′, respectively.That is, signals corresponding to each of the images surrounded by abroken line in FIG. 38 are inputted to each of the sub-pixel groups241′, 242′, 243′, and 244′.

Thereby, it becomes possible to display images of 4-viewpoints in totalof horizontally 2-viewpoints and vertically 2-viewpoints. At that time,there is no difference generated in the number of display colors amongeach of the viewpoint images within a pixel since all of the sub-pixelgroups 241′ to 244′ are constituted with the four primary colors.

Next, the relation between each of the viewpoint images and thesub-pixels in the second layout state will be described by referring toFIG. 37B and FIG. 38. As in the case of FIG. 37A, signals correspondingto the image B5 are inputted to the sub-pixel group 245′, signalscorresponding to the image B6 are inputted to the sub-pixel group 246′,signals corresponding to the image C5 are inputted to the sub-pixelgroup 247′, and signals corresponding to the image C6 are inputted tothe sub-pixel group 248′, respectively.

With such structure, the 3D resolution can be made equivalent for thefirst layout state and the second layout state regardless of the numberof viewpoints. That is, the number of viewpoint becomes the same in thefirst layout state and the second layout stat due to the regularity ofthe sub-pixel group within a single pixel. Thereby, same stereopsis canbe achieved in each of the layout states.

Based on the content described above, the relations regarding thefollowings are generally expressed bellow provided that: the number ofprimary colors is M; the viewpoint number in horizontal parallax displayis N (N is a natural number of 2 or larger); the horizontal viewpointnumber and the vertical viewpoint number in horizontal/vertical parallaxdisplay are L; the total viewpoint number of the horizontal/verticalparallax is J (=L×L); and the number of sub-pixels in a sub-pixel groupper direction in the horizontal/vertical parallax display is K.

Note here that the viewpoint number N is the number of viewpoints of acase of horizontal parallax display only, which corresponds to thenumber of sub-pixels per direction within a pixel.

First, K=√M when √M that is a square root of the primary color number Mis an integer, and K=M when √M is not an integer. For example, in a casewhere M=1, 4, 9, √M becomes an integer so that K=1, 2, 3. In a casewhere M=2, 3, √M does not become an integer so that K=2, 3.

Next, the horizontal viewpoint number and the vertical viewpoint numberL per direction is expressed as “L=N/K (where L is a natural number of 2or larger). For example, in a case of the pixel 221 shown in FIG. 31,“L=4/2=2”. As described, in the horizontal/vertical parallax display,employed is a structure with which the horizontal viewpoint number andthe vertical viewpoint number L per direction becomes a multiple of thesub-pixel number K in a sub-pixel group per direction in thehorizontal/vertical parallax display. This makes it possible to acquirean effect of having no color moire generated on the displayed image.

Next, FIG. 39 shows a state of light-ray separation in the x-axisdirection and the y-axis direction in the first layout state shown inFIG. 37A. In FIG. 39, the x-axis direction is the horizontal directionand the y-axis direction is the vertical direction.

Further, the light-ray separating module 250 for simultaneouslyseparating the light ray both in the first direction and the seconddirection is employed herein, so that parallax is given both in thehorizontal direction and the vertical direction.

FIG. 39 shows a state of a case where the observer moves to thepositions 114 a, 114 b, 114 c, - - - in the horizontal direction (x-axisdirection) and a state of a case where the observer moves to thepositions 115 a, 115 b, 115 c, 115 d, - - - in the vertical direction(y-axis direction) regarding the structure where the parallax is givenin both the first direction and the second direction.

The first-viewpoint region is constituted with the horizontal direction50 a and the vertical direction 55 a as the exit light from thesub-pixel group 241′, the second-viewpoint region is constituted withthe horizontal direction 50 b and the vertical direction 55 a as theexit light from the sub-pixel group 242′, the third-viewpoint region isconstituted with the horizontal direction 50 a and the verticaldirection 55 b as the exit light from the sub-pixel group 243′, and thefourth-viewpoint region is constituted with the horizontal direction 50b and the vertical direction 55 b as the exit light from the sub-pixelgroup 244′.

The main lobe in the horizontal direction is one region where the lefteye of the observer is located in 50 a and the right eye is located in50 b. In the meantime, in the vertical direction, there are two mainlobes where both eyes of the observer are located in 55 a or 55 b. Notehere that the region regarding 55 a is defined as a main lobe 1, theregion regarding 55 b is defined as a main lobe 2, the region regarding56 b is defined as a side lobe 1, and the region regarding 54 a isdefined as a side lobe 2.

FIG. 40 shows a corresponding relation regarding which of image data(see FIG. 38) is to be inputted to each viewpoint region (each sub-pixelgroup) when the left and right eyes of the observer are located at eachof the viewpoint regions with respect to each of the observer positions(see FIG. 39).

In general, when image data of 4-viewpoints are used for thestereoscopic image display panel of 2×2 viewpoints, only a same view issensed in the main lobe and the side lobes in the horizontal directionand reverse view is generated between the main lobe and the side lobes.In that case, similarly, only a same view is sensed between the mainlobe 1 and the side lobe 1 or between the main lobe 2 and the side lobe2, respectively, while no reverse view is generated in the verticaldirection.

Meanwhile, the use of the structure of the fourth exemplary embodimentmakes it possible to allow the observer to sense different videos in thestereoscopic regions A1, A2, A3, - - - , B1, B2, B3, - - - , C1, C2,C3, - - - , and D1, D2, D3, - - - even with the 2×2 viewpointsstereoscopic image display as shown in FIG. 40. That is, motion parallaxis given both in the horizontal and vertical directions, which cancontribute to improving a sense of ambience in the stereoscopic imagequality.

Note here that there is no binocular parallax in the vertical direction,it is also possible to employ a structure with which the separatingangle (viewpoint pitch) in the vertical direction becomes smaller withrespect to the value of the separating angle (viewpoint pitch) in thehorizontal direction.

As described above, in a case of using the active light-ray separatingmodule 250, it is possible with the fourth exemplary embodiment toemploy the structure with which light ray is separated only in thex-axis direction for the first layout state as in the case of FIG. 34,for example, and light ray is not separated in the y-axis direction andto employ the structure with which parallax is given both in thehorizontal direction and the vertical direction and light ray isseparated both in the x-axis direction and the y-axis directionsimultaneously as in the case of FIG. 39.

As described above, in a case of employing the structure where theseparating angle in the vertical direction is set smaller than theseparating angle in the horizontal direction, the lens power (refractionpower) of a lens 253 (FIG. 39) showing the state of separating the lightray in the vertical direction is designed to become smaller than thelens power of a lens 252 (FIG. 39) showing the state where light ray isseparated in the horizontal direction.

FIG. 41 shows an example of a terminal device 300 to which thestereoscopic image display device 1 according to the fourth exemplaryembodiment described above is mounted. That is, the terminal device 300is constituted with the stereoscopic image display device 1 and a casing310 which houses the device 1 inside thereof. The stereoscopic imagedisplay device 1 is provided with liquid crystal lens elements (notshown) capable of separating light ray in the directions along thex-axis direction and the y-axis direction as the light-ray separatingmodule 250 as described above.

Thus, the structure of the terminal device 300 to which the stereoscopicimage display device 1 according to the fourth exemplary embodiment willbe described by referring to FIG. 42.

With respect to the structure described in the first exemplaryembodiment, a detection unit 150 for detecting displacement caused whenthe terminal device 300 moves is added to the stereoscopic image displaydevice 1 herein. That is, the detection unit 150 is a structure foridentifying that the terminal device 300 is located horizontally orvertically for the observer.

The detection unit 150 is constituted by a sensor which detects thedisplacement caused when the terminal device 300 moves. Note here thatthe displacement of the terminal device 300 detected by the detectionunit 150 is a change in the tilt angle and a moving amount. For example,in a case where a sensor such as an acceleration sensor or a geomagneticsensor is used as the detection unit 150, the change in the tilt angleand the moving amount can be calculated on the basis of thegravitational acceleration and geomagnetism. That is, the detection unit150 includes a tilt angle detecting module 151 for detecting the changein the tilt angle and the moving amount so as to detect the positionalrelation between the stereoscopic image display panel 10 and theobserver.

Further, the detection unit 150 sends the information regarding the tiltangles (displacement information) as the detection result done by thetilt angle detecting module 151 to the image generation processing unit30.

That is, the image generation processing unit 30 is structured to outputthe image data according to the horizontal direction of the terminaldevice 300 and the observer position to the display panel drivingcircuit 21 according to the measurement result acquired from therelative position calculating unit 40 also shown in the first exemplaryembodiment described above and the signals (displacement information)from the detection unit 150.

When outputting 3D data, as described by referring to FIG. 32 and FIG.33, the sub-pixels corresponding to each viewpoint vary for the firstlayout state and the second layout state. Thus, the image generationprocessing unit 30 is structured to output the image data correspondingto the change to the display panel driving circuit 21.

Simultaneously with the output to the display panel driving circuit 21,the image generation processing unit 30 is structured to transmit acommand signal for making the light-ray separating module 250 (liquidcrystal lenses) effective in the x-axis direction and the y-axisdirection to the image distribution control circuit 22 according to eachof the layout states when the terminal device 300 is in the first layoutstate shown in FIG. 32 or in the second layout state shown in FIG. 33.

Further, for selecting the case of the N-viewpoint development only inthe horizontal direction as shown in the first exemplary embodiment(referred to as horizontal N-viewpoint development hereinafter) and thecase of developing L-viewpoints (L=N/K) each in the horizontal directionand the vertical direction (in total of J-viewpoints) as shown in thefourth exemplary embodiment (referred to as horizontal/verticalJ-viewpoint development hereinafter), an external IF 34 within the imagegeneration processing unit 30 is structured to function effectively. Forexample, it is possible to employ a structure with which a selectionsignal outputted from outside by a command or the like from the observeris received by the external IF 34, and the image generation processingunit 30 generates the image data based on the information contained inthe selection signal.

Therefore, the method for generating the image data by the imagegeneration processing unit 30 of the exemplary embodiment is a methodacquired by adding a method of changing the positions of the virtualcameras according to the selection signal received from outside to thegenerating method described in the first exemplary embodiment.

That is, the arithmetic calculator 31 sets the virtual cameras for theN-viewpoints only in the horizontal direction in the case of thehorizontal N-viewpoint development and sets the virtual cameras for theL-viewpoints (total of L×L viewpoints: J-viewpoints) in the horizontaldirection and the vertical direction in the case of thehorizontal/vertical J-viewpoint development. At the same time, thearithmetic calculator 31 is structured to generate the image data byperforming rendering processing in each of the cases.

Further, in the case of the horizontal N-viewpoint development, therelation of “(e+f)>d>e” described in the second exemplary embodiment canbe used when N=2. The relation of “e×(N-2)≦d≦e×(N-1)” described in thethird exemplary embodiment can be used when N≧3.

Like the contents described in each of those embodiments, when therelation is satisfied, it is possible with the structure of the fourthexemplary embodiment to provide the stereoscopic image display deviceexcellent in motion parallax while securing a prescribed stereoscopicvisibility.

Further, in the case of the horizontal/vertical J-viewpoint developmentshown in FIG. 37, the relation of “(e+f)>d>e” described in the secondexemplary embodiment can be used when J=2 (2×2 viewpoints for thehorizontal parallax direction). The relation of “e×(J-2)≦d≦e×(J-1)”described in the third exemplary embodiment can be used when J≧3. Thismakes it possible to provide the stereoscopic image display deviceexcellent in motion parallax while securing a prescribed stereoscopicvisibility.

For the vertical parallax direction, a smaller viewpoint pitch than thatof the horizontal parallax direction can be used. Thereby, it becomespossible to decrease color moire further and to give smooth motionparallax. Other operations are the same as those of the first exemplaryembodiment

(Explanations of Operations)

Next, operation contents of the stereoscopic image display device 1 andthe terminal device 300 according to the fourth exemplary embodiment canbe described briefly by referring to the flowchart shown in FIG. 43.

The detection unit 150 which has detected the displacement of theterminal device 300 outputs the displacement information as thedetection result thereof to the image generation processing unit 30(FIG. 43: S201).

Further, the observer position measuring unit 45 measures the positionof the observer 100 by using the camera provided inside thereof, andoutputs the measurement result to the relative position calculating unit(FIG. 43: S202).

Then, the relative position calculating unit 40 calculates the relativeposition of the left eye 101 and the right eye 102 with respect to thereference point (center of image display herein) set in advance on thestereoscopic image display panel 10 based on the measurement resultacquired from the observer position calculating unit 45, and outputs thecalculation result (positional information) (FIG. 43: S203).

Then, the image generation processing unit 30 generates and outputs 3Ddata and control signals based on the displacement information from thedetection unit 150 and the positional information from the relativeposition calculating unit 40 (FIG. 43: S204).

Subsequently, the display panel driving circuit 21 drives the displaypanel based on the 3D data acquired from the image generation processingunit 30, and the image distribution control circuit 22 controls theoperation of the light-separating module 12 according to the controlsignals from the image generation processing unit 30 (FIG. 43: S205).

For convenience, the operations are described in order of the numbersapplied in FIGS. 43 (S201 to S205). However, the operation content ofthe stereoscopic image display device 1 according to the fourthexemplary embodiment is not limited only to that order. Further, it isalso possible to put the execution content of each of theabove-described steps S201 to S205 (FIG. 43) into programs and have aseries of each of the control programs achieved by a computer.

Effects and the Like of Fourth Exemplary Embodiment

The stereoscopic image display device according to the fourth exemplaryembodiment employs the structure with which same numbers of parallaximages are generated in the x-axis direction and the y-axis directionwithin each pixel and the numbers of different color sub-pixels areequivalent in each viewpoint image. Thus, the 3D resolution can be madethe same for the first and second layout states in both of the caseswhere parallax is given in either the horizontal or the verticaldirection by the active light-ray separating module and where parallaxis given simultaneously for both of the two directions, i.e., thehorizontal direction and the vertical direction.

Further, the fourth exemplary embodiment is designed in such manner thatthe viewpoint number N becomes a multiple of the primary color number Min each pixel, so that generation of color moiré can be suppressed. Inaddition, while the state rotated clockwise by 90 degrees from the firstlayout state to set the y-axis direction as the horizontal direction isdescribed as the second layout state in the fourth exemplary embodiment,the second layout state is not limited only to that. For example, eachof the structural members can function effectively even in a state wherethe y-axis direction is set as the horizontal direction through rotatingthe first layout state by 90 degrees counterclockwise, for example, sothat the same operating effect as those of the second layout statedescribed herein can be acquired as well.

Other structures and operations are the same as those of the firstexemplary embodiment described above, and other operating effects to begenerated thereby are also the same. In a case where two or more imagedata are used for each viewpoint region in particular, smoother motionparallax can be presented both for horizontal parallax and verticalparallax.

Fifth Exemplary Embodiment

While the structural example of projecting the parallax images to theobserver by optically separating images towards two observing directionsis described in the fourth exemplary embodiment, such structural contentcan also be applied to the structures of the first to third exemplaryembodiments.

Therefore, in the fifth exemplary embodiment, a stereoscopic imagedisplay device acquired by applying the structural content for opticallyseparating images towards two observing directions to the structure ofthe first exemplary embodiment will be described by referring to FIG. 44and FIG. 45.

The overall structure of the stereoscopic image display device accordingto the fifth exemplary embodiment is the same as the structural contentof the stereoscopic image display device 1 shown in FIG. 1 and FIG. 2,so that each of those drawings is referred expediently for the followingexplanations. Therefore, same reference numerals are used for thestructural members and the like same as those of the first exemplaryembodiment, and duplicated explanations are to be omitted.

The stereoscopic image display panel 10 provided inside the stereoscopicimage display device 1 includes: a display panel 11 in which pixels eachcontaining at least a sub-pixel for displaying a first-viewpoint imageand a sub-pixel for displaying a second-viewpoint image are arranged inmatrix; and a light-ray separating module 12 which is provided on thedisplay surface side of the display panel 11 for separating parallaximages from the pixels towards each of the viewpoints.

As shown in FIG. 2, the light-ray separating module 12 has a structurein which lenses 15 corresponding to each of the pixels arranged on thedisplay panel 11 are arranged in an array form. That is, it is thestructure which separates light rays on one of the directions (light-rayseparating direction: x-direction of FIG. 44) of the layout directionsof each of the pixels.

Here, a direction in parallel to the straight line connecting betweenthe left eye and the right eye of the observer is defined as thehorizontal direction, and a case where the light-ray separatingdirection (x direction) is substantially in parallel to the horizontaldirection is assumed. Thus, the light-ray separating direction is alsoreferred to as the horizontal direction. Also, the direction vertical tothe horizontal direction (x direction) within a display surface (withina plane on parallel to the display surface) of the display panel 11 isdefined as the vertical direction (y direction of FIG. 44). The verticaldirection is the other direction out of the layout directions of each ofthe above-described pixels.

On the stereoscopic image display panel 10, a plurality of combinationsof a pixel 263 constituted with a first-viewpoint sub-pixel 261 and asecond-viewpoint sub-pixel 262 and the lens 15 constituting thelight-ray separating module provided at a position corresponding to thepixel 263 are disposed in matrix, and each of those is structured to beable to separate the light rays.

FIG. 44 is s view extracted from the structure shown in FIG. 2, whichshows the vicinity of the pixel containing sub-pixels for 2-viewpoints.Specifically, it shows a combination of the pixel 263 and the lens 15.Further, FIG. 44 shows an example of the state where correspondingimages are presented to each of the two directions according to theposition of the observer by using the unidirectional light-rayseparating module 12.

With the above-described combinations, images are separated in such amanner that the first-viewpoint regions 50 a, 51 a, (not shown), 52a, - - - to which the exit light from the sub-pixel group 261 spreadsand the second-viewpoint regions 50 b, 51 b, 52 b (not shown), - - - towhich the exit light from the sub-pixel group 262 are repeatedalternately along the x-axis direction.

Further, FIG. 44 also shows an example of the state of a case where theobserver moves to the positions 114 a, 114 b, 114 c, - - - in thehorizontal direction (x-axis direction) in such image separating spaceand the state of a case where observer moves to the positions 115 a, 115b, 115 c, 115 d, - - - in the vertical direction (y-axis direction).

In this case, the main lobe in the horizontal direction is one regionwhere the left eye of the observer is located in 50 a and the right eyeis located in 50 b. Meanwhile, since no lens effect exists in thevertical direction, there is no image separating space such as the mainlobe or the side lobes.

Next, FIG. 45 shows a corresponding relation regarding which of imagedata (see FIG. 38) is to be inputted to each viewpoint region (eachsub-pixel group) when the left and right eyes of the observer arelocated at each of the viewpoint regions with respect to each of theobserver positions (see FIG. 44).

As described, even with the stereoscopic image display panel whichseparates the image only in the horizontal direction of the 2-viewpoints(the stereoscopic image display panel which does not separate the imagein the vertical direction), it is possible to provide significant imagescorresponding to the moves of the observer in the horizontal/verticaldirections through switching and displaying the camera capturing imagesas appropriate according to the position of the observer in the verticaldirection.

Specifically, as shown in FIG. 45, through applying the image data A1,A2, A3 - - - in the vertical position 115 a, applying B1, B2, B3, - - -in the vertical position 115 b, applying C1, C2, C3, - - - in thevertical position 115 c, and applying D1, D2, D3, - - - in the verticalposition 115 d, it is possible to allow the observer to sense differentvideos as appropriate even when the observer moves in the verticaldirection. As a result, motion parallax is given both in the horizontaland vertical directions, which makes it possible to contribute toimproving a sense of ambience in the stereoscopic image quality.

Incidentally, as shown in FIG. 1, the stereoscopic image display device1 according to the fifth exemplary embodiment includes: the observerposition measuring unit 45 which measures the position of the observer;the relative position calculating unit 40 which calculates the relativeposition of the observer with respect to the stereoscopic image displaypanel 10 based on the measurement result; and the image generationprocessing unit 30 which executes generation of parallax images and thelike based on the relative position information calculated thereby.

Note here that the relative position information calculated by therelative position calculating unit 40 contains the positionalinformation of the observer in the horizontal and vertical directions.Thus, the image generation processing unit 30 can generate the image bytaking the move of the observer in the vertical direction intoconsideration based on the relative position information. That is,through outputting the generated image to the display panel 11 by theimage generation processing unit 30 via the display panel drivingcircuit 21, the switching processing of the images can be achieved.

The image switching angle in the horizontal direction (e.g., theswitching angle among A1, A2, A3, - - - in FIG. 38) is determined basedon the distance between the camera capturing positions in the horizontaldirection, and the image switching angle in the vertical direction(e.g., the switching angle among A1, B1, C1, D1 - - - in FIG. 38) isalso determined based on the distance between the camera capturingpositions in the vertical direction.

Through setting the image switching angle to be sufficiently smallerthan the viewpoint pitch, extremely smooth motion parallax can beacquired. In such case, it is particularly desirable to increase thecapacitance of the data accumulation unit 32 and the memory 33 withinthe display controller 24 according to the image switching angle and tospeed up the processing capacity of the arithmetic calculator 31.

While motion parallax to be given both in the horizontal and verticaldirections are described herein based on the structural content of thefirst exemplary embodiment, it is also possible to achieve the structurefor giving the motion parallax both in the horizontal and verticaldirections by a same method with the structural contents of the secondexemplary embodiment and the third exemplary embodiment as describedabove.

Effects and the Like of Fifth Exemplary Embodiment

As described above, the fifth exemplary embodiment employs the structurewith which: the light-ray separating module 12 has the structure forseparating the light ray in one direction (x direction) out of thelayout directions of each of the pixels; and the image generationprocessing unit 30 generates the image corresponding to the position ofthe observer regarding the direction (y direction) vertical to the onedirection described above within the display surface of the stereoscopicimage display panel 10 and outputs it to the stereoscopic image displaypanel 10. Therefore, the camera captured image according to the positionof the observer in the vertical direction can be presented to theobserver.

That is, even with the stereoscopic image display panel which separatesthe image only in the horizontal direction of 2-viewpoints, the use ofthe fifth exemplary embodiment makes it possible to present the cameracapturing images according to the position of the observer in thevertical direction. Therefore, motion parallax can be given both in thehorizontal and vertical directions, which makes it possible to improve asense of ambience in the stereoscopic images.

Further, through using the image data of the sufficiently smaller imageswitching angle than the viewpoint pitch based on the positionmeasurement information of the observer, it is possible to suppressdeterioration of the resolution and to acquire smooth motion parallaxeven with the multi-viewpoint stereoscopic image display device of smallnumber of viewpoints. Other structures and operations are the same asthose of each of the exemplary embodiments described above, and otheroperating effects to be generated thereby are also the same.

Each of the above-described embodiments shows specific preferableexamples of the stereoscopic image display device, the terminal device,the stereoscopic image display method, and the program thereof, andthere may be various kinds of technically preferable limits set thereto.However, the technical scope of the present invention is not limited tothose modes unless there is a specific remark mentioned for limiting thepresent invention.

While new technical contents regarding the above-described exemplaryembodiments are summarized as follows, the present invention is notnecessarily limited to those.

(Supplementary Note 1): First Exemplary Embodiment (J=s×N)

A stereoscopic image display device which includes:

a stereoscopic image display panel including a display panel in which aplurality of pixels are arranged and a light-ray separating moduleprovided on a display surface side of the display panel for separatingparallax images from each of the pixels towards a plurality ofN-viewpoints (N is a natural number of 2 or larger) according to layoutdirection of each of the pixels;

an observer position measuring unit which measures an observing positionof an observer who is facing the display surface;

a relative position calculating unit which calculates a relativeposition of the observer with respect to the stereoscopic image displaypanel based on a result of the measurement; and

an image generation processing unit which has a function for generatingviewpoint images for J-viewpoints (J>N) set in advance, and generatesone viewpoint image or more according to each of the viewpointsconstituting the N-viewpoints by corresponding to the relative positionand outputs the viewpoint image towards the stereoscopic image displaypanel.

(Supplementary Note 2): First Exemplary Embodiment (J=s×N)

The stereoscopic image display device as depicted in Supplementary Note1, wherein

the viewpoint number N and the viewpoint number J satisfy a relation ofJ=s×N (a coefficient s is a natural number of 2 or larger) (the relationof J=s×N (a coefficient s is a natural number of 2 or larger) existsbetween the N-viewpoints and the J-viewpoints).

(Supplementary Note 3): First Exemplary Embodiment (Side Lobes areAlmost Laterally Symmetric; s is an Odd Number of 3 or Larger)

The stereoscopic image display device as depicted in Supplementary Note2, wherein:

the stereoscopic image display panel has a lobe forming function whichforms a stereopsis region containing a display normal line that is anormal line from the display surface by having the center of imagedisplay as a start point as a main lobe and forms stereopsis regionsneighboring to the main lobe as side lobes;

provided that an optimum viewing distance at which a stereopsis visiblerange becomes maximum is OD and a viewpoint pitch between each ofviewpoint regions at the OD is e, the coefficient s is set in advance asan odd number of 3 or larger in a case where an angle formed between acenter line of the main lobe and the display normal line is equal to orless than a tan (e/OD); and

the image generation processing unit generates viewpoint images for theJ-viewpoints based on the set coefficient s.

(Supplementary Note 4): First Exemplary Embodiment (Side Lobes areAlmost Laterally Symmetric; s is an Odd Number of 3 or Larger)

The stereoscopic image display device as depicted in Supplementary Note2, wherein:

the stereoscopic image display panel has a lobe forming function whichforms a stereopsis region containing a display normal line that is anormal line from the display surface by having the center of imagedisplay as a start point as a main lobe and forms stereopsis regionsneighboring to the main lobe as side lobes;

the image generation processing unit has a function which calculates anangle formed between the center line of the main lobe and the displaynormal line based on the positional information and, provided that anoptimum viewing distance at which a stereopsis visible range becomesmaximum is OD and a viewpoint pitch between each of viewpoint regions atthe OD is e, the coefficient s is set in advance as an odd number of 3or larger, sets the coefficient s as an odd number of 3 or larger whenthe calculated angle is equal to or less than “a tan (e/OD)”.

(Supplementary Note 5): First Exemplary Embodiment (Select Image DataAccording to Moving Direction; N≧3)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 4, wherein:

when the viewpoint number N is 3 or larger (N≧3), the image generationprocessing unit selects a viewpoint image for a viewpoint region whereneither the left eye nor the right eye of the observer is located basedon a moving direction of the observer, and outputs the selectedviewpoint image towards the stereoscopic image display panel.

(Supplementary Note 6): First Exemplary Embodiment (Select Image DataAccording to Moving Direction; N≧3)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 4, wherein:

the image generation processing unit includes an image data switchingprocessing function which, when the viewpoint number N is 3 or larger(N≧3), estimates a next moving position of the observer based on themoving direction of the observer shown in the chronological positionalinformation from the relative position calculating unit and outputs theviewpoint images corresponding to the estimated position towards thestereoscopic image display panel.

(Supplementary Note 7): First Exemplary Embodiment (Select Image DataAccording to Moving Direction and Moving Speed; N≧4)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 4, wherein:

when the viewpoint number N is 4 or larger (N≧4), the image generationprocessing unit selects a viewpoint image for a viewpoint region whereneither the left eye nor the right eye of the observer is located basedon a moving direction and a moving speed of the observer, and outputsthe selected viewpoint image towards the stereoscopic image displaypanel.

(Supplementary Note 8): First Exemplary Embodiment (Select Image DataAccording to Moving Direction and Moving Speed; N≧4)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 4, wherein:

the image generation processing unit includes an image data switchingprocessing function which, when the viewpoint number N is 4 or larger(N≧4), estimates a next moving position of the observer based on themoving direction of the observer shown in the chronological positionalinformation from the relative position calculating unit and outputs theviewpoint images corresponding to the estimated position towards thestereoscopic image display panel.

(Supplementary Note 9): First Exemplary Embodiment (Select Image DataAccording to Moving Direction and Moving Speed; N≧4)

The stereoscopic image display device as depicted in any one ofSupplementary Note 7 or 8, wherein:

as the viewpoint region where neither the left eye nor the right eye ofthe observer is located, the image generation processing unit selectsfirst to (N-2)-th neighboring viewpoint regions by having the viewpointregion where the observer is located before moving as a start point.

(Supplementary Note 10): Second Exemplary Embodiment (Viewpoint Pitch ina Case of 2-Viewpoints)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 4, wherein

the stereoscopic image display panel is structured in such a manner thata viewpoint pitch between each of the viewpoint regions at the optimumobserving distance at which the stereopsis visible range becomes maximumbecomes smaller than a distance between both eyes of the observer, whenthe viewpoint number N is 2 (N=2).

(Supplementary Note 11): Second Exemplary Embodiment (Crosstalk Regionis Taken into Account)

The stereoscopic image display device as depicted in Supplementary Note10, wherein

the stereoscopic image display panel is structured to satisfy a relationof “(e+f)>d>e”, where the viewpoint pitch is e, the distance betweenboth eyes is d, and a range of the viewpoint region that is equal to orless than a 3D crosstalk value set in advance is f.

(Supplementary Note 12): Second Exemplary Embodiment (Crosstalk Regionis Taken into Account)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 4, wherein

the stereoscopic image display panel is structured to satisfy a relationof “(e+f)>d>e”, where the viewpoint pitch between each of the viewpointregions at the optimum viewing range where the stereoscopic viewingrange becomes the maximum is e, the distance between both eyes of theobserver is d, and a range of the viewpoint region that is equal to orless than a 3D crosstalk value set in advance is f.

(Supplementary Note 13): Third Exemplary Embodiment (Viewpoint Pitch inMulti-Viewpoints)

The stereoscopic image display device as depicted in any one of claims 1to 9, wherein

in a case where the viewpoint number N is 3 or larger, a viewpoint pitchbetween each of the viewpoint regions at the optimum viewing range wherethe stereopsis visible range becomes maximum is e, and a distancebetween both eyes of the observer is d, the stereoscopic image displaydevice is structured to satisfy a relation of “e×(N-2)≦d≦e×(N-1)”.

(Supplementary Note 14): Fourth Exemplary Embodiment (Vertical/Lateral3D)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 13, wherein:

each of the pixels includes N×N pieces (N is a natural number of 2 orlarger) of sub-pixels colored in M kinds (M is a natural number of 1 orlarger) primary colors, and the pixels are arranged in matrix on thedisplay panel;

the light-ray separating module includes optical elements whichdistribute exit light from each of the pixels according to the parallaximage to a first direction and a second direction along layout of thepixels;

the optical elements are arranged in matrix by being corresponded toeach of the pixels;

the sub-pixels neighboring to each other within each of the pixels areof different colors from each other;

layout pitches of the sub-pixels are uniform in the first direction andthe second direction; and

same numbers of the parallax images are displayed for the firstdirection and the second direction within each of the pixels, andnumbers of sub-pixels of different colors are equivalent within theparallax images.

(Supplementary Note 15): Fourth Exemplary Embodiment (Detection Unit)

The stereoscopic image display device depicted in Supplementary Note 14,which further includes a detection unit which detects displacement of amain body of the device including the display panel, wherein

the image generation processing unit has a function which generates theviewpoint images based on the relative position and displacementinformation from the detection unit.

(Supplementary Note 16): Fourth Exemplary Embodiment (Display inHorizontal and Vertical Directions)

The stereoscopic image display device as depicted in Supplementary Note15, wherein:

the detection unit has a function which judges whether it is in a firstlayout state where a horizontal direction that is a direction inparallel to a straight line connecting between the left eye and theright eye of the observer and the first direction are in parallel or asecond layout state where the horizontal direction and the seconddirection are in parallel; and

the image generation processing unit,

in a case where √M that is a square root of the M is an integer, takes aquotient (L) acquired by dividing the N with the √M as the viewpointnumber in the horizontal direction and in the vertical direction, andgenerates images having parallax for the viewpoint number according toeach of the first and second directions, and

in a case where √M that is a square root of the M that is the number ofprimary colors is not an integer, takes a quotient (L) acquired bydividing the N with the M as the viewpoint number in the horizontaldirection and in the vertical direction that is in an orthogonalrelation thereof, and generates images having parallax for the viewpointnumber according to each of the first and second directions.

(Supplementary Note 17): Fourth Exemplary Embodiment(Horizontal-Direction Viewpoint Pitch>Vertical-Direction ViewpointPitch)

The stereoscopic image display device as depicted in any one ofSupplementary Notes 14 to 16, wherein

when either the first direction or the second direction is in a statealong the horizontal direction that is a direction in parallel to thestraight line connecting between the left eye and the right eye of theobserver and the other is in a state along the vertical direction thatis vertical to the horizontal direction,

the stereoscopic image display panel is structured in such a manner thata parallax pitch in the horizontal direction becomes larger than aparallax pitch in the vertical direction.

(Supplementary Note 18): Fifth Exemplary Embodiment, etc.

The stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 13, wherein:

the light-ray separating module has a structure which separates lightray in one (x direction: light-ray separating direction) of thedirections out of the layout directions of each of the pixels; and

the image generation processing unit generates an image according to theposition of the observer regarding the other direction (Directionvertical to the one direction within the display surface: y direction)out of the layout directions of each of the pixels and outputs the imagetowards the stereoscopic image display panel.

(Supplementary Note 19) First to Fourth Exemplary Embodiments

A terminal device which includes:

the stereoscopic image display device as depicted in any one ofSupplementary Notes 1 to 18, and

a casing which houses the stereoscopic image display device insidethereof.

Supplementary Note 20

A stereoscopic image display method used with a stereoscopic imagedisplay device which includes: a stereoscopic image display panelincluding a display panel in which a plurality of pixels are arrangedand a light-ray separating module provided on a display surface side ofthe display panel for separating parallax images from each of the pixelstowards a plurality of N-viewpoints (N is a natural number of 2 orlarger) according to layout direction of each of the pixels; an observerposition measuring unit which measures an observing position of anobserver who is facing the display surface; and a display controllerwhich generates the parallax images according to a result of themeasurement, wherein:

the observer position measuring unit measures an observing position ofthe observer;

the display controller calculates a relative position of the observerwith respect to the stereoscopic image display panel based on the resultof the measurement;

the display controller generates viewpoint images for J-viewpoints (J>N)set in advance;

the display controller specifies a plurality of viewpoint imagesaccording to the relative position from the generated viewpoint images;and

the display controller outputs the specified plurality of viewpointimages towards the stereoscopic image display panel.

Supplementary Note 21

A stereoscopic image display method used with a stereoscopic imagedisplay device which includes: a stereoscopic image display panelincluding a display panel in which a plurality of pixels are arrangedand a light-ray separating module provided on a display surface side ofthe display panel for separating parallax images from each of the pixelstowards a plurality of N-viewpoints (N is a natural number of 2 orlarger) according to layout direction of each of the pixels; an observerposition measuring unit which measures an observing position of anobserver who is facing the display surface; and a display controllerwhich generates the parallax images according to a result of themeasurement, wherein:

the observer position measuring unit measures an observing position ofthe observer;

the display controller calculates a relative position of the observerwith respect to the stereoscopic image display panel based on the resultof the measurement;

the display controller specifies a plurality of viewpoints for emittingimages based on the relative position from J-viewpoints (J>N) set inadvance; and

the display controller generates viewpoint images according to thespecified plurality of viewpoints and outputs the images towards thestereoscopic image display panel.

Supplementary Note 22

A stereoscopic image display program used with a stereoscopic imagedisplay device which includes: a stereoscopic image display panelincluding a display panel in which a plurality of pixels are arrangedand a light-ray separating module provided on a display surface side ofthe display panel for separating parallax images from each of the pixelstowards a plurality of N-viewpoints (N is a natural number of 2 orlarger) according to layout direction of each of the pixels; an observerposition measuring unit which measures an observing position of anobserver who is facing the display surface; and a display controllerwhich generates the parallax images according to a result of themeasurement, and the program causes a computer provided in advance tothe display controller to function as:

an observer position measuring module which measures an observingposition of the observer;

an observer position calculating module which calculates a relativeposition of the observer with respect to the stereoscopic image displaypanel based on the result of the measurement; and

an image generation processing module which generates viewpoint imagesfor J-viewpoints (J>N) set in advance, specifies a plurality ofviewpoint images according to the relative position from the generatedviewpoint images, and outputs the specified plurality of viewpointimages towards the stereoscopic image display panel.

Supplementary Note 23

A stereoscopic image display program used with a stereoscopic imagedisplay device which includes: a stereoscopic image display panelincluding a display panel in which a plurality of pixels are arrangedand a light-ray separating module provided on a display surface side ofthe display panel for separating parallax images from each of the pixelstowards a plurality of N-viewpoints (N is a natural number of 2 orlarger) according to layout direction of each of the pixels; an observerposition measuring unit which measures an observing position of anobserver who is facing the display surface; and a display controllerwhich generates the parallax images according to a result of themeasurement, and the program causes a computer provided in advance tothe display controller to function as:

an observer position measuring module which measures an observingposition of the observer;

an observer position calculating module which calculates a relativeposition of the observer with respect to the stereoscopic image displaypanel based on the result of the measurement; and

an image generation processing module which specifies a plurality ofviewpoints for emitting images based on the relative position fromJ-viewpoints (J>N) set in advance, generates viewpoint images accordingto the specified plurality of viewpoints, and outputs the images towardsthe stereoscopic image display panel.

INDUSTRIAL APPLICABILITY

The stereoscopic image display device according to the present inventioncan be applied to various kinds of display devices which display images.

What is claimed is:
 1. A stereoscopic image display device, comprising:a stereoscopic image display panel comprising a display panel in which aplurality of pixels are arranged and a light-ray separating moduleprovided on a display surface side of the display panel for separatingparallax images from each of the pixels towards a plurality ofN-viewpoints (N is a natural number of 2 or larger) according to layoutdirection of each of the pixels; an observer position measuring unitwhich measures an observing position of an observer who is facing thedisplay surface; a relative position calculating unit which calculates arelative position of the observer with respect to the stereoscopic imagedisplay panel based on a result of the measurement; and an imagegeneration processing unit which has a function for generating viewpointimages for J-viewpoints (J>N) set in advance, and generates oneviewpoint image or more according to each of the viewpoints constitutingthe N-viewpoints by corresponding to the relative position and outputsthe viewpoint image towards the stereoscopic image display panel.
 2. Thestereoscopic image display device as claimed in claim 1, wherein theviewpoint number N and the viewpoint number J satisfy a relation ofJ=s×N (a coefficient s is a natural number of 2 or larger).
 3. Thestereoscopic image display device as claimed in claim 2, wherein: thestereoscopic image display panel has a lobe forming function which formsa stereopsis region containing a display normal line that is a normalline from the display surface by having the center of image display as astart point as a main lobe and forms stereopsis regions neighboring tothe main lobe as side lobes; provided that an optimum observing distanceat which a stereopsis visible range becomes maximum is OD and aviewpoint pitch between each of viewpoint regions at the OD is e, thecoefficient s is set in advance as an odd number of 3 or larger in acase where an angle formed between a center line of the main lobe andthe display normal line is equal to or less than a tan (e/OD); and theimage generation processing unit generates viewpoint images for theJ-viewpoints based on the set coefficient s.
 4. The stereoscopic imagedisplay device as claimed in claim 1, wherein: when the viewpoint numberN is 3 or larger, the image generation processing unit selects theviewpoint image for a viewpoint region where neither the left eye northe right eye of the observer is located based on a moving direction ofthe observer, and outputs the selected viewpoint image towards thestereoscopic image display panel.
 5. The stereoscopic image displaydevice as claimed in claim 1, wherein: when the viewpoint number N is 4or larger, the image generation processing unit selects the viewpointimage for a viewpoint region where neither the left eye nor the righteye of the observer is located based on a moving direction and a movingspeed of the observer, and outputs the selected viewpoint image towardsthe stereoscopic image display panel.
 6. The stereoscopic image displaydevice as claimed in claim 5, wherein: as the viewpoint region whereneither the left eye nor the right eye of the observer is located, theimage generation processing unit selects first to (N-2)-th neighboringviewpoint regions by having the viewpoint region where the observer islocated before moving as a start point.
 7. The stereoscopic imagedisplay device as claimed in claim 1, wherein the stereoscopic imagedisplay panel is structured in such a manner that a viewpoint pitchbetween each of the viewpoint regions at the optimum observing distanceat which the stereopsis visible range becomes maximum becomes smallerthan a distance between both eyes of the observer, when the viewpointnumber N is
 2. 8. The stereoscopic image display device as claimed inclaim 7, wherein the stereoscopic image display panel is structured tosatisfy a relation of “(e+f)>d>e”, where the viewpoint pitch is e, thedistance between both eyes is d, and a range of the viewpoint regionthat is equal to or less than a 3D crosstalk value set in advance is f.9. The stereoscopic image display device as claimed in claim 1, whereinin a case where the viewpoint number N is 3 or larger, a viewpoint pitchbetween each of the viewpoint regions at the optimum observing distanceat which the stereopsis visible range becomes maximum is e, and adistance between both eyes of the observer is d, the stereoscopic imagedisplay device is structured to satisfy a relation of“e×(N-2)≦d≦e×(N-1)”.
 10. The stereoscopic image display device asclaimed in claim 1, wherein: each of the pixels includes N×N pieces (Nis a natural number of 2 or larger) of sub-pixels colored in M kinds (Mis a natural number of 1 or larger) primary colors, and the pixels arearranged in matrix on the display panel; the light-ray separating moduleincludes optical elements which distribute exit light from each of thepixels according to the parallax image to a first direction and a seconddirection along layout of the pixels; the optical elements are arrangedin matrix by being corresponded to each of the pixels; the sub-pixelsneighboring to each other within each of the pixels are of differentcolors from each other; layout pitches of the sub-pixels are uniform inthe first direction and the second direction; and same numbers of theparallax images are displayed for the first direction and the seconddirection within each of the pixels, and numbers of sub-pixels ofdifferent colors are equivalent within the parallax images.
 11. Thestereoscopic image display device as claimed in claim 10, furthercomprising a detection unit which detects displacement of a main body ofthe device including the display panel, wherein the image generationprocessing unit has a function which generates the viewpoint imagesbased on the relative position and displacement information from thedetection unit.
 12. The stereoscopic image display device as claimed inclaim 11, wherein: the detection unit has a function which judgeswhether it is in a first layout state where a horizontal direction thatis a direction in parallel to a straight line connecting between theleft eye and the right eye of the observer and the first direction arein parallel or a second layout state where the horizontal direction andthe second direction are in parallel; and the image generationprocessing unit, in a case where √M that is a square root of the M thatis the number of primary colors is an integer, takes a quotient (L)acquired by dividing the N with the √M as the viewpoint number in thehorizontal direction and in a vertical direction that is in anorthogonal relation thereof, and generates images having parallax forthe viewpoint number according to each of the first and seconddirections, and in a case where √M that is a square root of the M is notan integer, takes a quotient (L) acquired by dividing the N with the Mas the viewpoint number in the horizontal direction and in the verticaldirection, and generates images having parallax for the viewpoint numberaccording to each of the first and second directions.
 13. Thestereoscopic image display device as claimed in claim 10, wherein wheneither the first direction or the second direction is in a state alongthe horizontal direction that is a direction in parallel to the straightline connecting between the left eye and the right eye of the observerand the other is in a state along a vertical direction that is verticalto the horizontal direction, the stereoscopic image display panel isstructured in such a manner that a parallax pitch in the horizontaldirection becomes larger than a parallax pitch in the verticaldirection.
 14. The stereoscopic image display device as claimed in claim1, wherein: the light-ray separating module has a structure whichseparates light ray in one of the directions out of the layoutdirections of each of the pixels; and the image generation processingunit generates an image according to the position of the observerregarding the other direction out of the layout directions of each ofthe pixels and outputs the image towards the stereoscopic image displaypanel.
 15. A terminal device, comprising: the stereoscopic image displaydevice as claimed in claim 1, and a casing which houses the stereoscopicimage display device inside thereof.
 16. A stereoscopic image displaymethod used with a stereoscopic image display device which comprises: astereoscopic image display panel comprising a display panel in which aplurality of pixels are arranged and a light-ray separating moduleprovided on a display surface side of the display panel for separatingparallax images from each of the pixels towards a plurality ofN-viewpoints (N is a natural number of 2 or larger) according to layoutdirection of each of the pixels; an observer position measuring unitwhich measures an observing position of an observer who is facing thedisplay surface; and a display controller which generates the parallaximages according to a result of the measurement, wherein: the observerposition measuring unit measures an observing position of the observer;the display controller calculates a relative position of the observerwith respect to the stereoscopic image display panel based on the resultof the measurement; the display controller generates viewpoint imagesfor J-viewpoints (J>N) set in advance; the display controller specifiesa plurality of viewpoint images according to the relative position fromthe generated viewpoint images; and the display controller outputs thespecified plurality of viewpoint images towards the stereoscopic imagedisplay panel.
 17. A stereoscopic image display method used with astereoscopic image display device which comprises: a stereoscopic imagedisplay panel comprising a display panel in which a plurality of pixelsare arranged and a light-ray separating module provided on a displaysurface side of the display panel for separating parallax images fromeach of the pixels towards a plurality of N-viewpoints (N is a naturalnumber of 2 or larger) according to layout direction of each of thepixels; an observer position measuring unit which measures an observingposition of an observer who is facing the display surface; and a displaycontroller which generates the parallax images according to a result ofthe measurement, wherein: the observer position measuring unit measuresan observing position of the observer; the display controller calculatesa relative position of the observer with respect to the stereoscopicimage display panel based on the result of the measurement; the displaycontroller specifies a plurality of viewpoints for emitting images basedon the relative position from J-viewpoints (J>N) set in advance; and thedisplay controller generates viewpoint images according to the specifiedplurality of viewpoints and outputs the images towards the stereoscopicimage display panel.